Local vs Remote Computation¶
The Fundamental Trade-off¶
Where should computation happen? Locally on your machine, or remotely on a server?
Local Computation:
┌─────────────────────────────────────────────────────────┐
│ Your Machine │
│ ┌──────────┐ ┌───────────┐ ┌──────────┐ │
│ │ Data │───▶│ Process │───▶│ Result │ │
│ └──────────┘ └───────────┘ └──────────┘ │
│ │
│ + No network latency │
│ + Data stays private │
│ - Limited by local hardware │
└─────────────────────────────────────────────────────────┘
Remote Computation:
┌──────────────────┐ ┌──────────────────┐
│ Your Machine │ │ Server │
│ ┌──────────┐ │ Network │ ┌───────────┐ │
│ │ Data │────┼───────────────────▶│ │ Process │ │
│ └──────────┘ │ (latency) │ └───────────┘ │
│ │ │ │ │
│ ┌──────────┐ │◀───────────────────┼────────┘ │
│ │ Result │◀───│ (latency) │ │
│ └──────────┘ │ │ + Powerful HW │
│ │ │ + Scalable │
│ - Network overhead │ - Latency cost │
└──────────────────┘ └──────────────────┘
Decision Factors¶
1. Data Size vs Computation Intensity¶
Data Size vs Computation:
Heavy computation, small data → Remote (use powerful servers)
Example: Training ML model on 1 MB dataset
Light computation, large data → Local (avoid transfer)
Example: Sum of 100 GB array
Heavy computation, large data → Complex decision
Example: Training on 1 TB dataset
2. Break-Even Analysis¶
def should_use_remote(data_size_gb, compute_time_local_s,
compute_time_remote_s, bandwidth_gbps=1):
"""Determine if remote computation is faster."""
# Transfer time (upload + download result)
transfer_time = data_size_gb / (bandwidth_gbps / 8) * 2
# Total remote time
total_remote = transfer_time + compute_time_remote_s
print(f"Local time: {compute_time_local_s:.1f}s")
print(f"Remote compute: {compute_time_remote_s:.1f}s")
print(f"Transfer overhead: {transfer_time:.1f}s")
print(f"Total remote: {total_remote:.1f}s")
return total_remote < compute_time_local_s
# Example: ML training
# Local laptop: 2 hours, Remote GPU: 10 minutes
# Data: 10 GB, Bandwidth: 100 Mbps
should_use_remote(
data_size_gb=10,
compute_time_local_s=7200, # 2 hours
compute_time_remote_s=600, # 10 minutes
bandwidth_gbps=0.1 # 100 Mbps
)
# Transfer: ~26 minutes
# Total remote: ~36 minutes
# Local: 120 minutes
# → Remote wins!
Scenarios and Recommendations¶
Scenario 1: Data Analysis¶
import pandas as pd
import numpy as np
# Small dataset analysis → Local
df = pd.read_csv('sales_100k.csv') # 10 MB
result = df.groupby('region').sum() # Fast locally
# Huge dataset → Consider remote or distributed
# 100 GB of data - transfer would take hours
# Options:
# 1. Remote SQL query (send query, not data)
# 2. Distributed processing (Spark, Dask)
# 3. Sample locally, full analysis on server
Scenario 2: Machine Learning¶
Training Decision Matrix:
┌─────────────────────────────────────────────────────────────┐
│ Small Model │ Large Model │
├─────────────────────────────────────────────────────────────┤
│ Small Data │ Local OK │ Remote (need GPU) │
│ │ (minutes) │ (hours) │
├───────────────────────────────┼─────────────────────────────┤
│ Large Data │ Remote │ Cloud (distributed) │
│ │ (data too big) │ (need cluster) │
└─────────────────────────────────────────────────────────────┘
# Local training - small model, small data
from sklearn.ensemble import RandomForestClassifier
model = RandomForestClassifier()
model.fit(X_train, y_train) # Local CPU fine
# Remote training - large model
# Send training job to cloud GPU
# Download trained model weights (small)
Scenario 3: API Services¶
# Remote makes sense for:
# - Complex NLP (GPT, BERT inference)
# - Image generation
# - Real-time translation
import openai
# Why remote?
# - Model is 100+ GB (can't run locally)
# - Requires specialized hardware
# - Latency acceptable for this use case
response = openai.ChatCompletion.create(
model="gpt-4",
messages=[{"role": "user", "content": "Hello!"}]
)
Scenario 4: Real-Time Processing¶
Low-latency requirements → Local
Gaming: Local (5ms latency max)
Video: Local decode, remote stream
Trading: Co-located (microseconds matter)
Hybrid Approaches¶
Edge Computing¶
Process close to data source, aggregate centrally:
Edge Computing Architecture:
┌─────────────────────────────────────────────────────────────┐
│ Cloud │
│ ┌────────────────────────────────────────────────────┐ │
│ │ Central Aggregation & Storage │ │
│ └────────────────────────────────────────────────────┘ │
│ ▲ ▲ ▲ │
└──────────────┼──────────────┼──────────────┼────────────────┘
│ │ │
┌─────────┴───┐ ┌──────┴────┐ ┌────┴──────┐
│ Edge │ │ Edge │ │ Edge │
│ Node 1 │ │ Node 2 │ │ Node 3 │
└──────┬──────┘ └─────┬─────┘ └─────┬─────┘
│ │ │
┌──────┴──────┐ ┌─────┴─────┐ ┌─────┴─────┐
│ Sensors │ │ Sensors │ │ Sensors │
│ (local proc)│ │ │ │ │
└─────────────┘ └───────────┘ └───────────┘
- Preprocess/filter locally
- Send summaries to cloud
- Reduces bandwidth by 10-100x
Data Locality¶
Move computation to data, not data to computation:
# Bad: Download all data, process locally
huge_data = remote_db.download_all() # 1 TB!
result = process(huge_data)
# Good: Run query on remote database
result = remote_db.query("""
SELECT region, SUM(sales)
FROM transactions
GROUP BY region
""")
# Only result transferred (kilobytes)
Tiered Processing¶
Tiered Architecture:
User Request
│
▼
┌─────────────────┐
│ Local Cache │ ← Check local first (fastest)
│ (in-memory) │
└────────┬────────┘
│ Miss
▼
┌─────────────────┐
│ Edge Server │ ← Check nearby server
│ (regional) │
└────────┬────────┘
│ Miss
▼
┌─────────────────┐
│ Origin Server │ ← Full computation
│ (central) │
└─────────────────┘
Cost Considerations¶
Cost Trade-offs:
Local:
+ No network costs
+ No cloud fees
- Hardware investment
- Power and cooling
- Maintenance
Remote:
+ No hardware investment
+ Elastic scaling
+ Latest hardware
- Per-use charges
- Data transfer costs
- Vendor dependency
Typical break-even:
Occasional use → Remote (pay-per-use)
Constant use → Local/dedicated (amortized cost)
Decision Framework¶
def choose_computation_location(
data_size_gb,
compute_intensity, # 'low', 'medium', 'high'
latency_requirement_ms,
privacy_sensitive,
frequency # 'one-time', 'daily', 'continuous'
):
"""Framework for local vs remote decision."""
# Hard constraints
if latency_requirement_ms < 10:
return "LOCAL (latency requirement)"
if privacy_sensitive and not compliant_remote_available():
return "LOCAL (privacy requirement)"
# Data size considerations
if data_size_gb > 100 and compute_intensity == 'low':
return "LOCAL (transfer cost too high)"
if data_size_gb < 1 and compute_intensity == 'high':
return "REMOTE (leverage powerful hardware)"
# Frequency considerations
if frequency == 'continuous':
if local_hardware_sufficient():
return "LOCAL (avoid ongoing costs)"
else:
return "DEDICATED REMOTE (reserved instances)"
# Default for occasional, moderate workloads
return "REMOTE (flexibility)"
Summary¶
| Factor | Favors Local | Favors Remote |
|---|---|---|
| Data Size | Large (transfer cost) | Small |
| Computation | Light | Heavy |
| Latency | Critical (<10ms) | Tolerant |
| Hardware | Sufficient locally | Need special (GPU) |
| Privacy | Sensitive | Not critical |
| Frequency | Continuous | Occasional |
| Cost | Owned hardware | Pay-per-use |
Key principles:
- Move computation to data when data is large
- Move data to computation when data is small and compute is specialized
- Cache aggressively to avoid repeated transfers
- Consider hybrid approaches for complex workloads
- Profile actual performance before deciding