5. HTTP & APIs

5. HTTP & APIs#

1. HTTP Status Codes - Complete Reference#

What are HTTP Status Codes?#

3-digit numbers sent by server in response to client request
Indicate what happened with the request
First digit indicates category

1xx - Informational#

Server received request, processing continues

100 Continue          → Client should continue sending request body
101 Switching         → Server agrees to switch protocols
    Protocols           (e.g., HTTP → WebSocket)
102 Processing        → Server processing, no response yet

2xx - Success ✅#

Request successfully received, understood, and accepted

200 OK                → Standard success ✅ most common
201 Created           → Resource created (after POST) ✅
202 Accepted          → Request accepted but not yet processed
204 No Content        → Success but no response body (DELETE) ✅
206 Partial Content   → Partial data returned (file download resume)

# Checking success in Python:
200 <= status < 300   # ✅ correct range check (exam answer)
status == 200         # ❌ misses 201, 204 etc.
status in [200, 201]  # ❌ incomplete
status < 400          # ❌ includes redirects

3xx - Redirection#

Further action needed to complete request

300 Multiple Choices  → Multiple options for resource
301 Moved Permanently → Resource permanently at new URL
302 Found             → Resource temporarily at different URL
303 See Other         → Redirect to another URL using GET
304 Not Modified      → Cached version still valid, no new data ✅
307 Temp Redirect     → Like 302 but must use same HTTP method
308 Perm Redirect     → Like 301 but must use same HTTP method

# Checking redirect in Python:
300 <= status < 400   # ✅ correct range check (exam answer)
status in [301, 302]  # ❌ incomplete
200 < status < 400    # ❌ includes some 2xx codes
status in range(299, 400) # ❌ starts at 299

4xx - Client Errors#

Request contains bad syntax or cannot be fulfilled

400 Bad Request       → Malformed syntax or invalid parameters
401 Unauthorized      → Authentication required/failed ✅ exam
                        (missing or invalid API key)
403 Forbidden         → Authenticated but no permission
404 Not Found         → Resource doesn't exist at URL
405 Method Not Allowed→ HTTP method not supported for endpoint
408 Request Timeout   → Server timed out waiting for request
409 Conflict          → Conflicts with current state (duplicate)
410 Gone              → Resource permanently deleted
413 Payload Too Large → Request body exceeds server limit
415 Unsupported       → Server doesn't support content type
    Media Type
422 Unprocessable     → Syntactically correct but semantically wrong
    Entity
429 Too Many Requests → Rate limit exceeded ✅ exam

5xx - Server Errors#

Server failed to fulfill valid request

500 Internal Server   → Generic server-side error ✅
    Error
501 Not Implemented   → Server doesn't support functionality
502 Bad Gateway       → Invalid response from upstream server
503 Service           → Server temporarily down
    Unavailable         (overloaded or maintenance)
504 Gateway Timeout   → Upstream server didn't respond in time
505 HTTP Version      → Server doesn't support HTTP version
    Not Supported

Key Distinctions - Exam Focus:#

401 vs 403:
→ 401 = not authenticated yet (no/wrong credentials)
→ 403 = authenticated but no permission

401 vs 429:
→ 401 = wrong/missing API key
→ 429 = valid key but too many requests

404 vs 410:
→ 404 = not found (may exist elsewhere)
→ 410 = permanently deleted

301 vs 302:
→ 301 = permanent redirect (update your bookmarks)
→ 302 = temporary redirect (keep old URL)

500 vs 503:
→ 500 = code error on server
→ 503 = server temporarily unavailable

502 vs 504:
→ 502 = bad response FROM upstream
→ 504 = upstream TIMED OUT

2. HTTP Methods - Complete Reference#

All HTTP Methods:#

GET     → Retrieve data (safe + idempotent)
POST    → Send/create data (not safe, not idempotent)
PUT     → Update/replace resource (idempotent)
PATCH   → Partial update (not idempotent)
DELETE  → Remove resource (idempotent)
HEAD    → Like GET but no body (just headers)
OPTIONS → Get allowed methods for resource

Safe vs Idempotent:#

Safe = does NOT modify server state
Idempotent = same result if called multiple times

         Safe    Idempotent
GET       ✅        ✅
HEAD      ✅        ✅
OPTIONS   ✅        ✅
POST      ❌        ❌
PUT       ❌        ✅
PATCH     ❌        ❌
DELETE    ❌        ✅

✅ GET = safe (does not modify server state) - exam answer
✅ PUT = idempotent (multiple identical requests = same effect) - exam answer
❌ POST = NOT idempotent (creates new resource each call)

GET - Retrieve Data:#

import requests

# Basic GET request
response = requests.get('https://api.openweathermap.org/data/2.5/weather',
    params={
        'q': 'Delhi',
        'appid': os.getenv('API_KEY')
    }
)

# Check status
print(response.status_code)    # 200
print(response.json())         # parsed JSON
print(response.text)           # raw text
print(response.headers)        # response headers

POST - Send Data / LLM Inference:#

# POST to LLM API (exam-relevant)
response = requests.post(
    'https://api.anthropic.com/v1/messages',
    headers={
        'x-api-key': os.getenv('ANTHROPIC_API_KEY'),
        'Content-Type': 'application/json'
    },
    json={
        'model': 'claude-3-sonnet-20240229',
        'max_tokens': 100,
        'messages': [
            {'role': 'user', 'content': 'Classify sentiment: Great product!'}
        ]
    }
)

data = response.json()
print(data['content'][0]['text'])

PUT - Update Resource:#

# PUT replaces entire resource
response = requests.put(
    'https://api.example.com/users/123',
    json={'name': 'John', 'email': 'john@example.com'}
)
# Calling same PUT 5 times = same result as calling once (idempotent)

PATCH - Partial Update:#

# PATCH updates only specified fields
response = requests.patch(
    'https://api.example.com/users/123',
    json={'email': 'newemail@example.com'}  # only update email
)

DELETE:#

response = requests.delete('https://api.example.com/users/123')
print(response.status_code)  # 204 No Content (success)

3. API Rate Limiting - Handling#

What is Rate Limiting?#

APIs limit number of requests per time period
Example: 60 calls/minute (OpenWeatherMap free tier)
Exceeding limit → HTTP 429 Too Many Requests

Strategies:#

1. Simple Sleep Between Requests:#

import requests
import time

cities = ['Delhi', 'Mumbai', 'Chennai', 'Kolkata', 'Bangalore']
results = []

for city in cities:
    response = requests.get(
        'https://api.openweathermap.org/data/2.5/weather',
        params={'q': city, 'appid': os.getenv('API_KEY')}
    )

    if response.status_code == 200:
        results.append(response.json())
    elif response.status_code == 429:
        print("Rate limit hit, waiting...")
        time.sleep(60)          # wait 1 minute
        # Retry
        response = requests.get(...)
        results.append(response.json())

    time.sleep(1)               # ✅ 1-2 sec between requests (exam answer)

print(f"Collected data for {len(results)} cities")

2. Exponential Backoff:#

import time
import requests

def fetch_with_retry(url, params, max_retries=3):
    """Retry with exponential backoff on rate limit"""
    for attempt in range(max_retries):
        response = requests.get(url, params=params)

        if response.status_code == 200:
            return response.json()

        elif response.status_code == 429:
            wait_time = 2 ** attempt  # 1, 2, 4 seconds
            print(f"Rate limited. Waiting {wait_time}s...")
            time.sleep(wait_time)

        elif response.status_code >= 500:
            wait_time = 2 ** attempt
            print(f"Server error {response.status_code}. Retrying...")
            time.sleep(wait_time)

        else:
            raise Exception(f"Unexpected status: {response.status_code}")

    raise Exception("Max retries exceeded")

3. Using `tenacity` Library:#

from tenacity import (
    retry,
    stop_after_attempt,
    wait_exponential,
    retry_if_exception_type
)

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, min=1, max=60)
)
def fetch_weather(city):
    response = requests.get(url, params={'q': city})
    if response.status_code == 429:
        raise Exception("Rate limited")
    return response.json()

✅ Use time.sleep(1-2) between requests - exam answer
❌ Make all requests simultaneously → triggers 429
❌ Ignore rate limit → API blocks your key
❌ Request only during off-peak hours → not reliable

4. API Security - Storing Keys#

Environment Variables - Best Practice:#

import os

# ✅ Read from environment variable
api_key = os.getenv('WEATHER_API_KEY')
api_key = os.environ.get('WEATHER_API_KEY')
api_key = os.getenv('WEATHER_API_KEY', 'default_value')  # with default

# Usage
response = requests.get(url, params={
    'q': 'Delhi',
    'appid': api_key      # ✅ from environment
})

Setting Environment Variables:#

# Linux/Mac - terminal
export WEATHER_API_KEY="your_key_here"
export ANTHROPIC_API_KEY="sk-ant-..."

# Windows - command prompt
set WEATHER_API_KEY=your_key_here

# Windows - PowerShell
$env:WEATHER_API_KEY = "your_key_here"

# Persistent - add to ~/.bashrc or ~/.zshrc
echo 'export WEATHER_API_KEY="your_key"' >> ~/.bashrc
source ~/.bashrc

Using `.env` File:#

# .env file (NEVER commit to git!)
WEATHER_API_KEY=your_key_here
ANTHROPIC_API_KEY=sk-ant-...
DATABASE_URL=postgresql://user:pass@localhost/db
DEBUG=false

# Load .env file with python-dotenv
from dotenv import load_dotenv
import os

load_dotenv()                              # loads .env file
api_key = os.getenv('WEATHER_API_KEY')    # read variable

# .gitignore - ALWAYS exclude .env
.env
.env.*
*.env
secrets.json

What NOT to Do:#

# ❌ Hard-coded in script (visible in git history)
api_key = "abc123xyz789"

# ❌ In code comments
# API key: abc123xyz789

# ❌ In GitHub README
# Use API key: abc123xyz789

# ❌ Printed to console (appears in logs)
print(f"Using key: {api_key}")

5. JSON Response Parsing - Nested Fields#

Understanding Nested JSON:#

# Typical API response structure
response_json = {
    "name": "Delhi",
    "main": {
        "temp": 298.15,
        "humidity": 60
    },
    "weather": [                    # ← this is a LIST
        {
            "description": "clear sky",
            "icon": "01d"
        }
    ],
    "wind": {
        "speed": 3.5
    }
}

# ✅ Correct access patterns:
city_name = response_json['name']                       # "Delhi"
temperature = response_json['main']['temp']             # 298.15
humidity = response_json['main']['humidity']            # 60
description = response_json['weather'][0]['description'] # ✅ "clear sky"
                                          # [0] needed - weather is a LIST
icon = response_json['weather'][0]['icon']              # "01d"
wind_speed = response_json['wind']['speed']             # 3.5

# ❌ Wrong access patterns:
response_json['weather']['description']        # ❌ weather is a list, not dict
response_json['main']['weather']               # ❌ wrong key
response_json['description']['weather']        # ❌ completely wrong order

# Safe access with .get() - avoids KeyError
temp = response_json.get('main', {}).get('temp', 0)

# Convert Kelvin to Celsius
temp_celsius = response_json['main']['temp'] - 273.15

6. `requests` Library - Complete Reference#

Installation & Import:#

pip install requests
import requests

All HTTP Methods:#

# GET
r = requests.get(url, params={'key': 'value'})

# POST
r = requests.post(url, json={'key': 'value'})
r = requests.post(url, data={'key': 'value'})  # form data

# PUT
r = requests.put(url, json={'key': 'value'})

# PATCH
r = requests.patch(url, json={'key': 'value'})

# DELETE
r = requests.delete(url)

# HEAD
r = requests.head(url)

Request Options:#

requests.get(
    url,
    params={'q': 'Delhi', 'appid': key},  # query parameters
    headers={
        'Authorization': f'Bearer {token}',
        'Content-Type': 'application/json',
        'User-Agent': 'MyApp/1.0'
    },
    json={'key': 'value'},      # request body (auto-sets Content-Type)
    data={'key': 'value'},      # form-encoded body
    timeout=30,                 # seconds before timeout
    verify=True,                # SSL certificate verification
    auth=('user', 'pass'),      # basic authentication
    cookies={'session': 'abc'}  # send cookies
)

Response Object:#

r = requests.get(url)

# Status
r.status_code               # 200, 404, etc.
r.ok                        # True if 200-299
r.raise_for_status()        # raises exception if 4xx/5xx ✅

# Content
r.text                      # response as string
r.json()                    # parse JSON response ✅
r.content                   # response as bytes
r.headers                   # response headers dict
r.url                       # final URL (after redirects)
r.history                   # redirect history
r.elapsed                   # time taken for request
r.cookies                   # response cookies

Error Handling:#

import requests
from requests.exceptions import (
    RequestException,
    ConnectionError,
    Timeout,
    HTTPError
)

def safe_api_call(url, params):
    try:
        response = requests.get(url, params=params, timeout=30)
        response.raise_for_status()     # raise for 4xx/5xx ✅
        return response.json()

    except requests.exceptions.Timeout:
        print("Request timed out")
        return None

    except requests.exceptions.ConnectionError:
        print("Connection failed")
        return None

    except requests.exceptions.HTTPError as e:
        print(f"HTTP error: {e.response.status_code}")
        if e.response.status_code == 401:
            print("Check your API key")
        elif e.response.status_code == 429:
            print("Rate limit exceeded, slow down")
        return None

    except requests.exceptions.RequestException as e:
        print(f"Request failed: {e}")
        return None

Sessions - Reuse Connection:#

# ✅ Use session for multiple requests to same API
# More efficient - reuses TCP connection

session = requests.Session()

# Set default headers for all requests
session.headers.update({
    'Authorization': f'Bearer {token}',
    'Content-Type': 'application/json'
})

# All requests use session settings
r1 = session.get(url1)
r2 = session.get(url2)
r3 = session.post(url3, json=data)

session.close()  # or use context manager:

with requests.Session() as session:
    session.headers.update({'Authorization': f'Bearer {token}'})
    r = session.get(url)

7. API Pagination#

What is Pagination?#

API returns data in pages (not all at once)
Need to make multiple requests to get all data

Common Pagination Patterns:#

1. Page Number Pagination:#

def get_all_results(base_url, params):
    """Fetch all pages of results"""
    all_results = []
    page = 1

    while True:
        params['page'] = page
        params['per_page'] = 100

        response = requests.get(base_url, params=params)
        data = response.json()

        results = data.get('results', [])
        if not results:
            break                   # no more pages

        all_results.extend(results)
        page += 1

        time.sleep(0.5)             # rate limiting

    return all_results

2. Cursor Pagination:#

def get_all_with_cursor(url, params):
    all_results = []
    cursor = None

    while True:
        if cursor:
            params['cursor'] = cursor

        response = requests.get(url, params=params)
        data = response.json()

        all_results.extend(data['items'])

        cursor = data.get('next_cursor')
        if not cursor:
            break                   # no more pages

    return all_results

3. Link Header Pagination (GitHub style):#

def get_all_github_results(url, headers):
    all_results = []

    while url:
        response = requests.get(url, headers=headers)
        all_results.extend(response.json())

        # GitHub puts next page URL in Link header
        link_header = response.headers.get('Link', '')
        next_url = None
        for part in link_header.split(','):
            if 'rel="next"' in part:
                next_url = part.split(';')[0].strip().strip('<>')
        url = next_url

    return all_results

8. Chrome DevTools - Network Tab for APIs#

What Network Tab Shows:#

Network Tab Features:
├── All HTTP requests made by page
├── Request details:
│   ├── Headers → request/response headers ✅
│   │   ├── Request URL
│   │   ├── Request Method (GET, POST)
│   │   ├── Status Code (200, 401, 429)
│   │   ├── Authorization header (API key location)
│   │   └── Content-Type
│   ├── Payload → request body (POST data)
│   ├── Response → actual response data ✅
│   │   └── JSON response body
│   └── Timing → request timing breakdown
│       ├── DNS lookup
│       ├── Connection time
│       ├── Time to First Byte (TTFB)
│       └── Content download
│
├── Filter by type:
│   └── XHR/Fetch → API calls specifically
│
└── Throttling → simulate slow network

Debugging API Issues:#

Problem: API call returns 401
→ Network tab → click request → Headers
→ Check: is Authorization header present?
→ Check: is API key format correct?
   Bearer token: "Authorization: Bearer sk-..."
   API key: "x-api-key: your-key"

Problem: Rate limiting (429)
→ Network tab → filter XHR
→ Look for pattern of 429 responses
→ Use Timing tab to see request frequency
→ Add sleep() between requests

Problem: Wrong data in response
→ Network tab → click request → Response tab
→ See exact JSON returned
→ Compare with expected structure

Complete API Workflow - Putting It Together:#

import requests
import time
import os
import json
from dotenv import load_dotenv

# Load environment variables
load_dotenv()

class WeatherAPIClient:
    def __init__(self):
        self.api_key = os.getenv('WEATHER_API_KEY')  # ✅ env variable
        self.base_url = 'https://api.openweathermap.org/data/2.5'
        self.session = requests.Session()
        self.session.params = {'appid': self.api_key}

    def get_weather(self, city, max_retries=3):
        """Fetch weather with error handling and retry"""
        url = f'{self.base_url}/weather'

        for attempt in range(max_retries):
            try:
                response = self.session.get(
                    url,
                    params={'q': city},
                    timeout=30
                )
                response.raise_for_status()   # raise on 4xx/5xx

                data = response.json()

                # ✅ Parse nested JSON correctly
                return {
                    'city': data['name'],
                    'temp_c': data['main']['temp'] - 273.15,
                    'humidity': data['main']['humidity'],
                    'description': data['weather'][0]['description']  # ✅ [0]
                }

            except requests.exceptions.HTTPError as e:
                if e.response.status_code == 401:
                    raise Exception("Invalid API key")  # don't retry

                elif e.response.status_code == 429:
                    wait = 2 ** attempt                 # exponential backoff
                    print(f"Rate limited. Waiting {wait}s")
                    time.sleep(wait)

                else:
                    raise

            except requests.exceptions.Timeout:
                time.sleep(2 ** attempt)

        raise Exception(f"Failed after {max_retries} attempts")

    def get_multiple_cities(self, cities):
        """Fetch weather for multiple cities with rate limiting"""
        results = []

        for city in cities:
            try:
                data = self.get_weather(city)
                results.append(data)
                print(f"✅ Got weather for {city}")

            except Exception as e:
                print(f"❌ Failed for {city}: {e}")

            time.sleep(1)       # ✅ rate limiting between requests

        return results

# Usage
client = WeatherAPIClient()
cities = ['Delhi', 'Mumbai', 'Chennai', 'Kolkata', 'Bangalore']
weather_data = client.get_multiple_cities(cities)

for city_data in weather_data:
    print(f"{city_data['city']}: {city_data['temp_c']:.1f}°C, {city_data['description']}")

HTTP & APIs - Quick Reference Card#

Status Codes:
  2xx → Success      (200 OK, 201 Created, 204 No Content)
  3xx → Redirect     (301 Permanent, 302 Temporary, 304 Cached)
  4xx → Client Error (400 Bad, 401 Auth ✅, 403 Forbidden,
                       404 Not Found, 429 Rate Limit ✅)
  5xx → Server Error (500 Internal, 503 Unavailable)

Range Checks:
  200 <= status < 300  → success ✅
  300 <= status < 400  → redirect ✅

HTTP Methods:
  GET    → retrieve (safe + idempotent) ✅
  POST   → create/send (LLM inference) ✅
  PUT    → replace (idempotent) ✅
  DELETE → remove (idempotent)
  PATCH  → partial update

Rate Limiting:
  429 → rate limit exceeded
  Fix: time.sleep(1-2) between requests ✅
  Advanced: exponential backoff

API Security:
  ✅ os.getenv('API_KEY')
  ✅ .env file + python-dotenv
  ❌ Hard-code in script
  ❌ Commit to git

JSON Parsing:
  response.json()                         # parse response
  data['weather'][0]['description']       # nested + list ✅
  data.get('key', default)                # safe access

5. HTTP & APIs#

1. HTTP Status Codes - Complete Reference#

What are HTTP Status Codes?#

1xx - Informational#

2xx - Success ✅#

3xx - Redirection#

4xx - Client Errors#

5xx - Server Errors#

Key Distinctions - Exam Focus:#

2. HTTP Methods - Complete Reference#

All HTTP Methods:#

Safe vs Idempotent:#

GET - Retrieve Data:#

POST - Send Data / LLM Inference:#

PUT - Update Resource:#

PATCH - Partial Update:#

DELETE:#

3. API Rate Limiting - Handling#

What is Rate Limiting?#

Strategies:#

1. Simple Sleep Between Requests:#

2. Exponential Backoff:#

3. Using tenacity Library:#

4. API Security - Storing Keys#

Environment Variables - Best Practice:#

Setting Environment Variables:#

Using .env File:#

What NOT to Do:#

5. JSON Response Parsing - Nested Fields#

Understanding Nested JSON:#

6. requests Library - Complete Reference#

Installation & Import:#

All HTTP Methods:#

Request Options:#

Response Object:#

Error Handling:#

Sessions - Reuse Connection:#

7. API Pagination#

What is Pagination?#

Common Pagination Patterns:#

1. Page Number Pagination:#

2. Cursor Pagination:#

3. Link Header Pagination (GitHub style):#

8. Chrome DevTools - Network Tab for APIs#

What Network Tab Shows:#

Debugging API Issues:#

Complete API Workflow - Putting It Together:#

HTTP & APIs - Quick Reference Card#

3. Using `tenacity` Library:#

Using `.env` File:#

6. `requests` Library - Complete Reference#