PyTorch Hello World

Introduction

Welcome to the PyTorch Hello World tutorial! If you're just starting with PyTorch, you're in the right place. PyTorch is a powerful open-source machine learning library developed by Facebook's AI Research lab. It's widely used for applications like deep learning and natural language processing.

In traditional programming, a "Hello World" example demonstrates the simplest possible working code. For PyTorch, our "Hello World" will involve creating tensors - the fundamental data structure in PyTorch.

By the end of this tutorial, you'll understand:

• What tensors are
• How to install PyTorch
• How to create and manipulate tensors
• How to perform basic operations with PyTorch

Prerequisites

Before we begin, make sure you have:

• Python 3.x installed
• Basic knowledge of Python programming
• A code editor or Jupyter notebook environment

Installing PyTorch

Let's start by installing PyTorch. Open your terminal or command prompt and run:

bash

pip install torch torchvision

This command installs both PyTorch and torchvision (a package with computer vision datasets and models).

tip

For specific installation instructions based on your operating system and CUDA requirements, visit the official PyTorch website.

Understanding Tensors

Tensors are the core data structure in PyTorch. Think of them as multi-dimensional arrays or matrices that can run on GPUs to accelerate computing.

• A 0-dimensional tensor is a scalar (single value)
• A 1-dimensional tensor is a vector (like a list)
• A 2-dimensional tensor is a matrix (rows and columns)
• A tensor with more dimensions represents more complex data structures

Your First PyTorch Code

Let's create our first tensor to say "Hello World" with PyTorch:

python

import torch

# Create a scalar (0-dimensional tensor)
scalar = torch.tensor(42)
print("Scalar tensor:", scalar)
print("Dimension:", scalar.dim())

# Create a vector (1-dimensional tensor)
vector = torch.tensor([1, 2, 3, 4])
print("\nVector tensor:", vector)
print("Dimension:", vector.dim())
print("Shape:", vector.shape)

# Create a matrix (2-dimensional tensor)
matrix = torch.tensor([[1, 2], [3, 4], [5, 6]])
print("\nMatrix tensor:")
print(matrix)
print("Dimension:", matrix.dim())
print("Shape:", matrix.shape)

# Create a "Hello World" message using ASCII values
hello_world = torch.tensor([72, 101, 108, 108, 111, 32, 87, 111, 114, 108, 100])
hello_message = ''.join([chr(i.item()) for i in hello_world])
print("\nDecoded message:", hello_message)

Output:

Scalar tensor: tensor(42)
Dimension: 0

Vector tensor: tensor([1, 2, 3, 4])
Dimension: 1
Shape: torch.Size([4])

Matrix tensor:
tensor([[1, 2],
        [3, 4],
        [5, 6]])
Dimension: 2
Shape: torch.Size([3, 2])

Decoded message: Hello World

Basic Tensor Operations

Now that you've created tensors, let's explore some basic operations:

Creating Tensors with Special Properties

python

import torch

# Create tensors with specific values
zeros = torch.zeros(3, 4)  # 3x4 tensor of zeros
ones = torch.ones(2, 2)    # 2x2 tensor of ones
random_tensor = torch.rand(2, 3)  # 2x3 tensor with random values

print("Zeros tensor:")
print(zeros)
print("\nOnes tensor:")
print(ones)
print("\nRandom tensor:")
print(random_tensor)

# Create tensor with specific data type
int_tensor = torch.ones(2, 2, dtype=torch.int)
print("\nInteger tensor:")
print(int_tensor)

Output:

Zeros tensor:
tensor([[0., 0., 0., 0.],
        [0., 0., 0., 0.],
        [0., 0., 0., 0.]])

Ones tensor:
tensor([[1., 1.],
        [1., 1.]])

Random tensor:
tensor([[0.1234, 0.5678, 0.9012],
        [0.3456, 0.7890, 0.2345]])

Integer tensor:
tensor([[1, 1],
        [1, 1]], dtype=torch.int32)

Mathematical Operations

python

import torch

# Create two tensors
a = torch.tensor([1, 2, 3])
b = torch.tensor([4, 5, 6])

# Addition
print("Addition:", a + b)  # Element-wise addition
print("Addition (method):", torch.add(a, b))

# Subtraction
print("Subtraction:", a - b)

# Multiplication (element-wise)
print("Element-wise multiplication:", a * b)

# Matrix multiplication
m1 = torch.tensor([[1, 2], [3, 4]])
m2 = torch.tensor([[5, 6], [7, 8]])
print("\nMatrix 1:\n", m1)
print("Matrix 2:\n", m2)
print("Matrix multiplication:\n", torch.matmul(m1, m2))

Output:

Addition: tensor([5, 7, 9])
Addition (method): tensor([5, 7, 9])
Subtraction: tensor([-3, -3, -3])
Element-wise multiplication: tensor([4, 10, 18])

Matrix 1:
 tensor([[1, 2],
        [3, 4]])
Matrix 2:
 tensor([[5, 6],
        [7, 8]])
Matrix multiplication:
 tensor([[19, 22],
        [43, 50]])

Moving to GPU (if available)

One of PyTorch's strengths is its seamless GPU support. Here's how to check if a GPU is available and move tensors to it:

python

import torch

# Check if GPU is available
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
print(f"Using device: {device}")

# Create a tensor
x = torch.tensor([1, 2, 3])
print(f"Tensor created on: {x.device}")

# Move tensor to appropriate device
x = x.to(device)
print(f"Tensor moved to: {x.device}")

Output (will vary based on your hardware):

Using device: cpu
Tensor created on: cpu
Tensor moved to: cpu

If you have a CUDA-compatible GPU and PyTorch is installed with CUDA support, the output would show "cuda:0" instead of "cpu".

Real-World Application: Simple Neural Network

Let's put our knowledge to use with a minimal neural network example:

python

import torch
import torch.nn as nn
import torch.optim as optim

# Create a simple dataset: y = 2x
x = torch.linspace(-5, 5, 100).reshape(-1, 1)
y = 2 * x

# Define a simple model
class SimpleModel(nn.Module):
    def __init__(self):
        super(SimpleModel, self).__init__()
        self.linear = nn.Linear(1, 1)  # Input dim: 1, Output dim: 1
        
    def forward(self, x):
        return self.linear(x)

# Initialize the model
model = SimpleModel()
print("Initial model parameters:", model.linear.weight.item(), model.linear.bias.item())

# Define loss function and optimizer
criterion = nn.MSELoss()
optimizer = optim.SGD(model.parameters(), lr=0.01)

# Training loop
for epoch in range(100):
    # Forward pass
    y_pred = model(x)
    loss = criterion(y_pred, y)
    
    # Backward pass and optimization
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    
    if (epoch + 1) % 10 == 0:
        print(f'Epoch {epoch+1}/100, Loss: {loss.item():.4f}')

print("Final model parameters:", model.linear.weight.item(), model.linear.bias.item())
print("Expected weight: 2.0, Expected bias: 0.0")

Output:

Initial model parameters: 0.4913892149925232 -0.39015659689903259
Epoch 10/100, Loss: 0.5658
Epoch 20/100, Loss: 0.0608
Epoch 30/100, Loss: 0.0065
Epoch 40/100, Loss: 0.0007
Epoch 50/100, Loss: 0.0001
Epoch 60/100, Loss: 0.0000
Epoch 70/100, Loss: 0.0000
Epoch 80/100, Loss: 0.0000
Epoch 90/100, Loss: 0.0000
Epoch 100/100, Loss: 0.0000
Final model parameters: 1.9999048709869385 -0.0006310975697636604
Expected weight: 2.0, Expected bias: 0.0

In this example, we:

1. Created a simple dataset where y = 2x
2. Built a single-layer neural network
3. Trained the model to learn the relationship between x and y
4. Observed how the weight converges to 2 and the bias to 0

Summary

Congratulations! You've completed the PyTorch Hello World tutorial. Here's what you've learned:

• PyTorch is a powerful library for machine learning and deep learning
• Tensors are the fundamental data structure in PyTorch
• You can create tensors of different dimensions and types
• PyTorch supports various mathematical operations on tensors
• Tensors can be moved between CPU and GPU for accelerated computing
• You can build and train neural networks using PyTorch

This is just the beginning of your PyTorch journey! As you continue, you'll discover how to build more complex models and solve challenging problems in machine learning and artificial intelligence.

Additional Resources

To deepen your understanding of PyTorch, check out these resources:

• Official PyTorch Documentation
• PyTorch Tutorials
• PyTorch GitHub Repository

Practice Exercises

1. Create a 3-dimensional tensor with random values between 0 and 1
2. Write a function that normalizes a tensor (scales values to be between 0 and 1)
3. Build a simple linear regression model to fit a different equation (e.g., y = 3x + 1)
4. Experiment with different optimization algorithms (like Adam instead of SGD)
5. Create and train a simple classifier on the MNIST dataset using PyTorch