Python Requests Library: Making HTTP API Calls for Absolute Beginners

Making HTTP requests is one of the most common tasks in modern programming. Whether you're fetching data from an API, submitting forms, or downloading files, Python's requests library makes it incredibly simple and intuitive.

💡

🎯 What You'll Learn: In this comprehensive guide, you'll discover:

How to run Python scripts correctly (shebang vs python command)
Installing Python packages with pip
Making HTTP GET requests with the requests library
Understanding HTTP status codes and what they mean
Working with JSON responses from APIs
Parsing and accessing data from API responses
Common mistakes and how to avoid them
Best practices for working with HTTP requests

🚀 Understanding Python Script Execution

Before we dive into making HTTP requests, let's understand the difference between running Python scripts with ./script.py versus python script.py.

Creating Your First Python Script

Let's create a simple Python script that will make an HTTP request:

import requests

response = requests.get("https://api.github.com")

print(response.status_code)

What this script does:

import requests - Imports the requests library
requests.get("https://api.github.com") - Makes an HTTP GET request to GitHub's API
print(response.status_code) - Prints the HTTP status code of the response

Prerequisites

Before we dive in, make sure you have:

Python 3.x installed on your system
Basic understanding of Python syntax
Terminal/command line access
Internet connection (to make API calls)

⚠️ Common Mistake: Using ./script.py

After creating the script, you might try to run it like a shell script:

chmod +x req.py
./req.py

Result:

./req.py: line 1: import: command not found
./req.py: line 3: syntax error near unexpected token `('
./req.py: line 3: `response = requests.get("https://api.github.com")'

Why This Fails

What happened:

chmod +x req.py - Made the file executable
./req.py - Attempted to run it directly
Error: Bash tried to execute it as a shell script, not a Python script

Why it failed:

When you run ./req.py, your shell (bash) looks for a shebang line at the top of the file
A shebang line looks like #!/usr/bin/python3 or #!/usr/bin/env python3
Without a shebang, bash tries to interpret the Python code as shell commands
The word import is not a bash command, hence the error

⚠️

⚠️ Important: Python code uses syntax that bash doesn't understand. The ( parentheses, = assignments, and import statement are all Python-specific syntax that causes bash to throw errors.

The Correct Way to Run Python Scripts

There are two proper ways to run Python scripts:

Method 1: Using the python command (Recommended for beginners)

python req.py

python3 req.py

Method 2: Adding a shebang line

Add this as the first line of your script:

#!/usr/bin/env python3

Then you can run:

chmod +x req.py
./req.py

For this tutorial, we'll use python req.py as it's more explicit and beginner-friendly.

📦 Installing the Requests Library

Before making HTTP requests, we need to install the requests library. Python doesn't include it by default.

Using pip to Install Packages

pip install requests

Output:

Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: requests in /usr/lib/python3.9/site-packages (2.25.1)
Requirement already satisfied: chardet<5,>=3.0.2 in /usr/lib/python3.9/site-packages (from requests) (4.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/lib/python3.9/site-packages (from requests) (1.26.5)
Requirement already satisfied: idna<3,>=2.5 in /usr/lib/python3.9/site-packages (from requests) (2.10)

Understanding pip install Output

Let's break down what this output means:

Line 1: Defaulting to user installation because normal site-packages is not writeable

What it means: pip is installing the package in your user directory instead of system-wide
Why: You don't have administrator/root permissions to write to system directories
Impact: The package is available only for your user account
Is this a problem?: No, this is perfectly normal and safe

Line 2: Requirement already satisfied: requests in /usr/lib/python3.9/site-packages (2.25.1)

What it means: The requests library (version 2.25.1) is already installed
Location: /usr/lib/python3.9/site-packages
Action taken: pip skipped installation since it's already present

Lines 3-5: Dependencies already satisfied

chardet - Character encoding detection library
urllib3 - HTTP library (requests uses this internally)
idna - Internationalized Domain Names support

These are dependencies: Libraries that requests needs to function properly. pip automatically checks and installs these when you install requests.

✅

✅ Package Management: pip automatically handles dependencies for you. When you install a package, pip installs all the libraries it needs to work correctly.

🌐 Making Your First HTTP Request

Now that we have the requests library installed, let's run our script:

python req.py

Output:

What this means:

The script successfully connected to https://api.github.com
GitHub's API server responded with status code 200
Status code 200 means "OK" - the request was successful

Understanding the Code

Let's examine each line of our script:

import requests

Purpose: Imports the requests library into your script

What it does: Makes all the functions and classes from the requests library available for use in your code

Why we need it: Python only loads basic functionality by default. To use external libraries, we must import them first.

response = requests.get("https://api.github.com")

Purpose: Makes an HTTP GET request to the specified URL

Breaking it down:

requests.get() - Function that sends an HTTP GET request
"https://api.github.com" - The URL we're requesting
response - Variable that stores the response object returned by the API

What happens behind the scenes:

Your script sends a GET request to GitHub's server
GitHub's server processes the request
GitHub sends back a response with data
The response is stored in the response variable

print(response.status_code)

Purpose: Prints the HTTP status code from the response

What it does:

Accesses the status_code attribute of the response object
Displays it on the screen

📊 HTTP Status Codes Explained

HTTP status codes are three-digit numbers that indicate the result of an HTTP request. They're grouped into five categories:

Category	Range	Meaning	Example
Informational	100-199	Request received, continuing process	100 Continue
Success	200-299	Request successfully received and accepted	200 OK, 201 Created
Redirection	300-399	Further action needed to complete request	301 Moved Permanently
Client Error	400-499	Request contains errors or cannot be fulfilled	404 Not Found, 403 Forbidden
Server Error	500-599	Server failed to fulfill valid request	500 Internal Server Error

Common HTTP Status Codes

Code	Name	What It Means
`200`	OK	Request succeeded, data returned successfully
`201`	Created	Request succeeded, new resource created
`204`	No Content	Request succeeded but no data to return
`301`	Moved Permanently	Resource permanently moved to new URL
`400`	Bad Request	Server cannot understand the request
`401`	Unauthorized	Authentication required and failed or not provided
`403`	Forbidden	Server understood but refuses to authorize
`404`	Not Found	Requested resource doesn't exist on server
`429`	Too Many Requests	User sent too many requests in given time (rate limiting)
`500`	Internal Server Error	Server encountered unexpected condition
`502`	Bad Gateway	Server got invalid response from upstream server
`503`	Service Unavailable	Server temporarily unable to handle request

📝 Working with JSON Responses

APIs typically return data in JSON (JavaScript Object Notation) format. Let's explore how to work with JSON responses.

Mistake: Forgetting Parentheses

Let's modify our script to print the JSON data. First, let's see a common mistake:

import requests

response = requests.get("https://api.github.com")

print(response.json)

Running this:

python req.py

Output:

<bound method Response.json of <Response [200]>>

What Went Wrong?

Why this output:

response.json - References the method object itself (without calling it)
In Python, methods are objects too
Without (), you're printing the method reference, not calling it
<bound method ...> indicates it's a method waiting to be called

Think of it like this:

response.json - "Show me the function"
response.json() - "Run the function and give me the result"

⚠️

⚠️ Common Python Mistake: Forgetting parentheses when calling methods is a very common error. Always use () to actually execute a function or method.

The Correct Way: Using Parentheses

Update the script:

import requests

response = requests.get("https://api.github.com")

print(response.json())

Running this:

python req.py

Output (formatted for readability):

{
  'current_user_url': 'https://api.github.com/user',
  'current_user_authorizations_html_url': 'https://github.com/settings/connections/applications{/client_id}',
  'authorizations_url': 'https://api.github.com/authorizations',
  'code_search_url': 'https://api.github.com/search/code?q={query}{&page,per_page,sort,order}',
  'commit_search_url': 'https://api.github.com/search/commits?q={query}{&page,per_page,sort,order}',
  'emails_url': 'https://api.github.com/user/emails',
  'emojis_url': 'https://api.github.com/emojis',
  'events_url': 'https://api.github.com/events',
  'feeds_url': 'https://api.github.com/feeds',
  'followers_url': 'https://api.github.com/user/followers',
  'following_url': 'https://api.github.com/user/following{/target}',
  'gists_url': 'https://api.github.com/gists{/gist_id}',
  # ... and many more API endpoints
}

Understanding the JSON Response

What is response.json():

Method call: json() is a method of the response object
Purpose: Parses the JSON string returned by the API into a Python dictionary
Return value: A Python dictionary containing the API data

What the output shows:

This is a Python dictionary (notice the single quotes around keys)
Each key-value pair represents an available API endpoint
GitHub's root API provides a directory of all available endpoints
The URLs contain placeholders like {query} that you'd replace with actual values

Example endpoint breakdown:

'user_url': 'https://api.github.com/users/{user}'

Key: 'user_url' - Name of the endpoint
Value: 'https://api.github.com/users/{user}' - URL template
Usage: Replace {user} with an actual username to get user information

🔍 Parsing and Accessing JSON Data

Now that we have a Python dictionary, let's learn how to access specific data from it.

Accessing Dictionary Values

import requests

response = requests.get("https://api.github.com")
data = response.json()

# Access specific endpoints
print("User URL template:", data['user_url'])
print("Repository search URL:", data['repository_search_url'])
print("Emojis URL:", data['emojis_url'])

Output:

User URL template: https://api.github.com/users/{user}
Repository search URL: https://api.github.com/search/repositories?q={query}{&page,per_page,sort,order}
Emojis URL: https://api.github.com/emojis

What's happening:

data = response.json() - Stores the dictionary in a variable
data['user_url'] - Accesses the value associated with the 'user_url' key
Dictionary access uses square brackets [] with the key name in quotes

Pretty Printing JSON

For better readability, use Python's json module:

import requests
import json

response = requests.get("https://api.github.com")
data = response.json()

# Pretty print with indentation
print(json.dumps(data, indent=2))

What this does:

json.dumps() - Converts Python dictionary back to JSON string
indent=2 - Adds 2-space indentation for readability
Output is formatted with proper line breaks and indentation

Sample formatted output:

{
  "current_user_url": "https://api.github.com/user",
  "authorizations_url": "https://api.github.com/authorizations",
  "emails_url": "https://api.github.com/user/emails",
  "emojis_url": "https://api.github.com/emojis"
}

🔧 Response Object Attributes and Methods

The response object contains much more than just the data. Let's explore its useful attributes and methods:

Essential Response Attributes

import requests

response = requests.get("https://api.github.com")

# Status information
print("Status Code:", response.status_code)
print("OK?:", response.ok)  # True if status code is less than 400

# Headers
print("\nContent Type:", response.headers['Content-Type'])
print("Server:", response.headers.get('Server'))

# Response content
print("\nResponse text length:", len(response.text))
print("Response encoding:", response.encoding)

# URL information
print("\nRequested URL:", response.url)

# Timing
print("Response time:", response.elapsed.total_seconds(), "seconds")

Sample output:

Status Code: 200
OK?: True

Content Type: application/json; charset=utf-8
Server: GitHub.com

Response text length: 2234
Response encoding: utf-8

Requested URL: https://api.github.com

Response time: 0.156789 seconds

Understanding Response Attributes

Attribute/Method	Type	Description
`status_code`	int	HTTP status code (200, 404, etc.)
`ok`	bool	True if status code is less than 400
`text`	str	Response content as text/string
`content`	bytes	Response content as bytes (for binary data)
`json()`	dict/list	Parse JSON response into Python object
`headers`	dict	Response headers (metadata)
`url`	str	Final URL of the response
`encoding`	str	Character encoding (utf-8, etc.)
`elapsed`	timedelta	Time taken for the request
`raise_for_status()`	None	Raises exception if status code indicates error

🚨 Error Handling

Always handle potential errors when making HTTP requests:

import requests

try:
    response = requests.get("https://api.github.com", timeout=5)
    response.raise_for_status()  # Raise exception for 4xx/5xx status codes

    data = response.json()
    print("Success! Received", len(data), "API endpoints")

except requests.exceptions.Timeout:
    print("Error: Request timed out")

except requests.exceptions.ConnectionError:
    print("Error: Failed to connect to the server")

except requests.exceptions.HTTPError as e:
    print(f"HTTP Error: {e}")

except requests.exceptions.JSONDecodeError:
    print("Error: Response is not valid JSON")

except Exception as e:
    print(f"Unexpected error: {e}")

Why error handling matters:

Network requests can fail for many reasons
Servers can be down or unreachable
API responses might not be in expected format
Timeouts can occur with slow connections
Proper error handling prevents your program from crashing

🎯 Practical Example: Fetching User Data

Let's create a practical script that fetches GitHub user information:

import requests
import json

def get_github_user(username):
    """Fetch GitHub user information"""
    url = f"https://api.github.com/users/{username}"

    try:
        response = requests.get(url, timeout=10)
        response.raise_for_status()

        user_data = response.json()

        # Display user information
        print(f"\n{'='*50}")
        print(f"GitHub User: {user_data['login']}")
        print(f"{'='*50}")
        print(f"Name: {user_data.get('name', 'N/A')}")
        print(f"Bio: {user_data.get('bio', 'N/A')}")
        print(f"Public Repos: {user_data['public_repos']}")
        print(f"Followers: {user_data['followers']}")
        print(f"Following: {user_data['following']}")
        print(f"Location: {user_data.get('location', 'N/A')}")
        print(f"Profile: {user_data['html_url']}")
        print(f"{'='*50}\n")

    except requests.exceptions.HTTPError:
        if response.status_code == 404:
            print(f"Error: User '{username}' not found")
        else:
            print(f"HTTP Error: {response.status_code}")
    except Exception as e:
        print(f"Error: {e}")

# Test with a username
get_github_user("torvalds")  # Linus Torvalds

Sample output:

==================================================
GitHub User: torvalds
==================================================
Name: Linus Torvalds
Bio: N/A
Public Repos: 6
Followers: 180543
Following: 0
Location: Portland, OR
Profile: https://github.com/torvalds
==================================================

🎯 Best Practices

✅ Making HTTP Requests

Always use timeouts: Prevent indefinite hanging with timeout parameter
Handle errors gracefully: Use try-except blocks for network errors
Check status codes: Verify the request succeeded before processing data
Use response.raise_for_status(): Automatically raise exceptions for error codes
Close connections: Requests library handles this automatically, but be aware for long-running scripts
Respect rate limits: Many APIs limit how many requests you can make per hour

✅ Working with JSON

Use response.json(): Easier than manually parsing with json.loads()
Check Content-Type: Ensure the response is actually JSON before calling .json()
Use .get() for optional keys: Prevents KeyError if key doesn't exist
Pretty print during development: Use json.dumps(data, indent=2) for readability
Validate data structure: Check that expected keys exist before accessing them

✅ Security and Privacy

Never hardcode credentials: Use environment variables for API keys
Use HTTPS: Always use secure connections for sensitive data
Validate input: Sanitize user input before using in URLs
Be careful with sensitive data: Don't log or print API keys or passwords
Check SSL certificates: Requests verifies by default, don't disable without good reason

✅ Performance

Reuse sessions: Use requests.Session() for multiple requests to same host
Enable connection pooling: Session objects do this automatically
Use streaming for large files: stream=True parameter for downloads
Set appropriate timeouts: Balance between reliability and speed
Consider async for many requests: Use aiohttp for concurrent requests

📝 Command and Concept Cheat Sheet

Installing Packages

# Install a package
pip install requests

# Install specific version
pip install requests==2.28.0

# Upgrade a package
pip install --upgrade requests

# Uninstall a package
pip uninstall requests

# List installed packages
pip list

# Show package information
pip show requests

Running Python Scripts

# Using python command
python script.py
python3 script.py

# Using shebang (add #!/usr/bin/env python3 to top of file)
chmod +x script.py
./script.py

Basic HTTP Requests

import requests

# GET request
response = requests.get("https://api.example.com/data")

# GET with parameters
params = {"q": "search term", "page": 1}
response = requests.get("https://api.example.com/search", params=params)

# GET with timeout
response = requests.get("https://api.example.com", timeout=5)

# POST request
data = {"username": "john", "email": "john@example.com"}
response = requests.post("https://api.example.com/users", json=data)

# Custom headers
headers = {"Authorization": "Bearer YOUR_TOKEN"}
response = requests.get("https://api.example.com/private", headers=headers)

Working with Responses

# Status code
print(response.status_code)

# Check if request was successful
if response.ok:
    print("Success!")

# Response text
print(response.text)

# Parse JSON
data = response.json()

# Response headers
print(response.headers)
print(response.headers['Content-Type'])

# Response time
print(response.elapsed.total_seconds())

# Raise exception for error status codes
response.raise_for_status()

Error Handling Template

import requests

try:
    response = requests.get(url, timeout=10)
    response.raise_for_status()
    data = response.json()
    # Process data

except requests.exceptions.Timeout:
    print("Request timed out")
except requests.exceptions.ConnectionError:
    print("Connection failed")
except requests.exceptions.HTTPError as e:
    print(f"HTTP error: {e}")
except requests.exceptions.JSONDecodeError:
    print("Invalid JSON response")
except Exception as e:
    print(f"Error: {e}")

HTTP Status Code Quick Reference

# Success codes
200  # OK - Request succeeded
201  # Created - Resource created successfully
204  # No Content - Success but no data to return

# Redirection codes
301  # Moved Permanently
302  # Found (temporary redirect)
304  # Not Modified (cached version is valid)

# Client error codes
400  # Bad Request - Invalid syntax
401  # Unauthorized - Authentication required
403  # Forbidden - No permission
404  # Not Found - Resource doesn't exist
429  # Too Many Requests - Rate limited

# Server error codes
500  # Internal Server Error
502  # Bad Gateway
503  # Service Unavailable

🚀 What's Next?

📚 Continue Learning

Now that you understand HTTP requests with Python, explore:

POST, PUT, DELETE requests: Modify data on servers
Authentication: Working with API keys, OAuth, and tokens
Request sessions: Reusing connections for better performance
Async requests: Making concurrent requests with aiohttp
Web scraping: Extracting data from HTML pages with BeautifulSoup
API pagination: Handling large datasets split across multiple pages
File uploads: Sending files through HTTP requests

✅

🎉 Congratulations! You've learned how to make HTTP requests in Python using the requests library. You now understand status codes, JSON parsing, and best practices for working with APIs!

What did you think of this guide? Share your questions or your own API projects in the comments below!

💬 Discussion

I'd love to hear about your experience:

What APIs are you planning to work with?
Have you encountered any interesting API challenges?
What other HTTP/API topics would you like to learn about?
Share any cool projects you're building with the requests library!

Connect with me:

🐙 GitHub - API examples and more
📧 Contact - Python and API discussions

🚀 Understanding Python Script Execution

Creating Your First Python Script

Prerequisites

⚠️ Common Mistake: Using ./script.py

Why This Fails

The Correct Way to Run Python Scripts

📦 Installing the Requests Library

Using pip to Install Packages

Understanding pip install Output

🌐 Making Your First HTTP Request

Understanding the Code

📊 HTTP Status Codes Explained

Common HTTP Status Codes

📝 Working with JSON Responses

Mistake: Forgetting Parentheses

What Went Wrong?

The Correct Way: Using Parentheses

Understanding the JSON Response

🔍 Parsing and Accessing JSON Data

Accessing Dictionary Values

Pretty Printing JSON

🔧 Response Object Attributes and Methods

Essential Response Attributes

Understanding Response Attributes

🚨 Error Handling

🎯 Practical Example: Fetching User Data

🎯 Best Practices

✅ Making HTTP Requests

✅ Working with JSON

✅ Security and Privacy

✅ Performance

📝 Command and Concept Cheat Sheet

Installing Packages

Running Python Scripts

Basic HTTP Requests

Working with Responses

Error Handling Template

HTTP Status Code Quick Reference

🚀 What's Next?

📚 Continue Learning

💬 Discussion

Written by Owais

Related Articles

Python JSON Handling: Complete Guide to Data Serialization and File Operations

Python CSV Handling for Absolute Beginners: A Complete Terminal Tutorial with csv.reader and csv.DictReader

Python Lambda Functions, Map, Filter, Reduce & Comprehensions: Complete Beginner's Guide with Terminal Examples

More Reading

Python Recursion vs Iteration: Performance, Memory Usage, and Tail Recursion Explained