TensorFlow Tensors

Introduction

In TensorFlow, tensors are the fundamental data structure that forms the backbone of all computations. Simply put, tensors are multi-dimensional arrays that can hold data with uniform type. If you're familiar with NumPy arrays, TensorFlow tensors are quite similar but with added capabilities that make them optimized for deep learning operations.

Why are they called "tensors"? The name comes from mathematics, where tensors represent geometric or physical quantities. In machine learning, they represent the data that flows through your neural networks – hence the name "TensorFlow."

In this tutorial, you'll learn:

• What tensors are and how they work in TensorFlow
• How to create different types of tensors
• Basic operations you can perform with tensors
• How to manipulate tensor shapes and dimensions
• Practical applications of tensors in machine learning

What Are Tensors?

A tensor is a generalization of vectors and matrices to potentially higher dimensions. Tensors can be represented by n-dimensional arrays:

• A scalar is a 0-dimensional tensor (just a single number)
• A vector is a 1-dimensional tensor (a list of numbers)
• A matrix is a 2-dimensional tensor (a table of numbers)
• A 3D tensor could represent a cube of numbers
• And so on to higher dimensions

Every tensor in TensorFlow has:

1. A data type (like float32, int32, string)
2. A shape that defines the number of dimensions and the size of each dimension

Creating Tensors in TensorFlow

Let's start by importing TensorFlow and creating some basic tensors:

python

import tensorflow as tf
import numpy as np

# Create a simple scalar (0D tensor)
scalar = tf.constant(42)
print("Scalar tensor:", scalar)
print("Shape:", scalar.shape)
print("Data type:", scalar.dtype)

# Create a vector (1D tensor)
vector = tf.constant([1, 2, 3, 4, 5])
print("\nVector tensor:", vector)
print("Shape:", vector.shape)

# Create a matrix (2D tensor)
matrix = tf.constant([[1, 2, 3], [4, 5, 6]])
print("\nMatrix tensor:", matrix)
print("Shape:", matrix.shape)

# Create a 3D tensor
tensor_3d = tf.constant([[[1, 2], [3, 4]], [[5, 6], [7, 8]]])
print("\n3D tensor:\n", tensor_3d)
print("Shape:", tensor_3d.shape)

Output:

Scalar tensor: tf.Tensor(42, shape=(), dtype=int32)
Shape: ()
Data type: <dtype: 'int32'>

Vector tensor: tf.Tensor([1 2 3 4 5], shape=(5,), dtype=int32)
Shape: (5,)

Matrix tensor: tf.Tensor(
[[1 2 3]
 [4 5 6]], shape=(2, 3), dtype=int32)
Shape: (2, 3)

3D tensor:
 tf.Tensor(
[[[1 2]
  [3 4]]

 [[5 6]
  [7 8]]], shape=(2, 2, 2), dtype=int32)
Shape: (2, 2, 2)

Creating Special Tensors

TensorFlow provides functions to create tensors with special values:

python

# Tensor filled with zeros
zeros = tf.zeros([3, 4])
print("Zeros tensor:\n", zeros)

# Tensor filled with ones
ones = tf.ones([2, 3])
print("\nOnes tensor:\n", ones)

# Tensor with a specific value
filled = tf.fill([2, 3], 42)
print("\nFilled tensor:\n", filled)

# Random tensors
random_uniform = tf.random.uniform([3, 3], minval=0, maxval=10)
print("\nRandom uniform tensor:\n", random_uniform)

random_normal = tf.random.normal([3, 3], mean=0, stddev=1)
print("\nRandom normal tensor:\n", random_normal)

# Creating a range
range_tensor = tf.range(start=0, limit=10, delta=2)
print("\nRange tensor:", range_tensor)

# Creating tensor from NumPy array
numpy_array = np.array([[1, 2], [3, 4]])
numpy_tensor = tf.convert_to_tensor(numpy_array)
print("\nNumPy to tensor:\n", numpy_tensor)

Output:

Zeros tensor:
 tf.Tensor(
[[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]], shape=(3, 4), dtype=float32)

Ones tensor:
 tf.Tensor(
[[1. 1. 1.]
 [1. 1. 1.]], shape=(2, 3), dtype=float32)

Filled tensor:
 tf.Tensor(
[[42 42 42]
 [42 42 42]], shape=(2, 3), dtype=int32)

Random uniform tensor:
 tf.Tensor(
[[1.6562529 9.8496485 3.3179295]
 [0.13066673 9.127702  0.5940573]
 [8.302584   2.080879  3.1970797]], shape=(3, 3), dtype=float32)

Random normal tensor:
 tf.Tensor(
[[ 0.7047504  -0.10664198 -1.1581349 ]
 [ 1.5323266  -1.6120005  -0.5426969 ]
 [-0.32200432 -1.2049222   0.04493868]], shape=(3, 3), dtype=float32)

Range tensor: tf.Tensor([0 2 4 6 8], shape=(5,), dtype=int32)

NumPy to tensor:
 tf.Tensor(
[[1 2]
 [3 4]], shape=(2, 2), dtype=int32)

Tensor Attributes and Operations

Once you have created tensors, you can access various attributes and perform operations on them.

Basic Tensor Properties

python

# Create a sample tensor
sample = tf.constant([[1, 2, 3], [4, 5, 6]])

# Get shape
print("Shape:", sample.shape)

# Get rank (number of dimensions)
print("Rank:", tf.rank(sample))

# Get size (total number of elements)
print("Size:", tf.size(sample))

# Get data type
print("Data type:", sample.dtype)

# Convert to NumPy array
numpy_version = sample.numpy()
print("NumPy array:\n", numpy_version)

Output:

Shape: (2, 3)
Rank: tf.Tensor(2, shape=(), dtype=int32)
Size: tf.Tensor(6, shape=(), dtype=int32)
Data type: <dtype: 'int32'>
NumPy array:
 [[1 2 3]
 [4 5 6]]

Basic Operations with Tensors

You can perform arithmetic operations on tensors just like with regular numbers:

python

a = tf.constant([[1, 2], [3, 4]])
b = tf.constant([[5, 6], [7, 8]])

# Addition
print("Addition:\n", a + b)  # Or tf.add(a, b)

# Subtraction
print("\nSubtraction:\n", a - b)  # Or tf.subtract(a, b)

# Element-wise multiplication
print("\nElement-wise multiplication:\n", a * b)  # Or tf.multiply(a, b)

# Matrix multiplication
print("\nMatrix multiplication:\n", tf.matmul(a, b))

# Division
print("\nDivision:\n", a / b)  # Or tf.divide(a, b)

# Element-wise square root
print("\nSquare root:\n", tf.sqrt(tf.cast(a, tf.float32)))

# Exponentiation
print("\nExponentiation (a^2):\n", tf.pow(a, 2))

Output:

Addition:
 tf.Tensor(
[[ 6  8]
 [10 12]], shape=(2, 2), dtype=int32)

Subtraction:
 tf.Tensor(
[[-4 -4]
 [-4 -4]], shape=(2, 2), dtype=int32)

Element-wise multiplication:
 tf.Tensor(
[[ 5 12]
 [21 32]], shape=(2, 2), dtype=int32)

Matrix multiplication:
 tf.Tensor(
[[19 22]
 [43 50]], shape=(2, 2), dtype=int32)

Division:
 tf.Tensor(
[[0.2        0.33333334]
 [0.42857143 0.5       ]], shape=(2, 2), dtype=float64)

Square root:
 tf.Tensor(
[[1.        1.4142135]
 [1.7320508 2.       ]], shape=(2, 2), dtype=float32)

Exponentiation (a^2):
 tf.Tensor(
[[ 1  4]
 [ 9 16]], shape=(2, 2), dtype=int32)

Manipulating Tensor Shapes

Reshaping tensors is a common operation in deep learning pipelines, especially when preparing data for different model architectures.

python

# Create a tensor with 12 elements
t = tf.constant([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11])
print("Original tensor:", t)
print("Shape:", t.shape)

# Reshape to a 3x4 matrix
reshaped = tf.reshape(t, [3, 4])
print("\nReshaped to 3x4:\n", reshaped)

# Reshape to a 2x2x3 tensor
reshaped_3d = tf.reshape(t, [2, 2, 3])
print("\nReshaped to 2x2x3:\n", reshaped_3d)

# Change the order of dimensions (transpose)
matrix = tf.constant([[1, 2, 3], [4, 5, 6]])
transposed = tf.transpose(matrix)
print("\nOriginal matrix:\n", matrix)
print("\nTransposed matrix:\n", transposed)

# Adding a new dimension
expanded = tf.expand_dims(matrix, axis=0)
print("\nExpanded tensor (added dimension at axis 0):\n", expanded)
print("New shape:", expanded.shape)

# Removing a dimension with size 1
squeezed = tf.squeeze(expanded)
print("\nSqueezed tensor (removed dimension with size 1):\n", squeezed)
print("New shape:", squeezed.shape)

Output:

Original tensor: tf.Tensor([ 0  1  2  3  4  5  6  7  8  9 10 11], shape=(12,), dtype=int32)
Shape: (12,)

Reshaped to 3x4:
 tf.Tensor(
[[ 0  1  2  3]
 [ 4  5  6  7]
 [ 8  9 10 11]], shape=(3, 4), dtype=int32)

Reshaped to 2x2x3:
 tf.Tensor(
[[[ 0  1  2]
  [ 3  4  5]]

 [[ 6  7  8]
  [ 9 10 11]]], shape=(2, 2, 3), dtype=int32)

Original matrix:
 tf.Tensor(
[[1 2 3]
 [4 5 6]], shape=(2, 3), dtype=int32)

Transposed matrix:
 tf.Tensor(
[[1 4]
 [2 5]
 [3 6]], shape=(3, 2), dtype=int32)

Expanded tensor (added dimension at axis 0):
 tf.Tensor(
[[[1 2 3]
  [4 5 6]]], shape=(1, 2, 3), dtype=int32)
New shape: (1, 2, 3)

Squeezed tensor (removed dimension with size 1):
 tf.Tensor(
[[1 2 3]
 [4 5 6]], shape=(2, 3), dtype=int32)
New shape: (2, 3)

Slicing and Indexing Tensors

You can extract specific parts of a tensor using slicing operations:

python

# Create a sample tensor
sample_matrix = tf.constant([
    [1, 2, 3, 4],
    [5, 6, 7, 8],
    [9, 10, 11, 12]
])

# Get a single element
print("Element at [0, 2]:", sample_matrix[0, 2].numpy())

# Get the first row
print("\nFirst row:", sample_matrix[0].numpy())

# Get the second column
print("\nSecond column:", sample_matrix[:, 1].numpy())

# Slicing [start:stop:step]
print("\nFirst two rows:", sample_matrix[0:2].numpy())

# Slicing with steps
print("\nEvery other element in the first row:", sample_matrix[0, ::2].numpy())

# Advanced slicing - get a submatrix
print("\nSubmatrix (first 2 rows, last 2 columns):\n", sample_matrix[0:2, 2:4].numpy())

Output:

Element at [0, 2]: 3

First row: [1 2 3 4]

Second column: [ 2  6 10]

First two rows: [[1 2 3 4]
 [5 6 7 8]]

Every other element in the first row: [1 3]

Submatrix (first 2 rows, last 2 columns):
 [[3 4]
 [7 8]]

Practical Example: Image Processing with Tensors

Let's see how tensors can be used for a real-world application like image processing. We'll load an image, convert it to a tensor, and perform some basic operations:

python

import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np

# Load an image (assuming you have one in the current directory)
# If you don't have an image, comment this section out
try:
    # Load an image using TensorFlow's IO operations
    image_raw = tf.io.read_file('sample_image.jpg')
    image = tf.io.decode_jpeg(image_raw, channels=3)
    
    # Convert to float32 and normalize
    image = tf.cast(image, tf.float32) / 255.0
    
    print("Image tensor shape:", image.shape)
    print("Data type:", image.dtype)
    
    # Display the image
    plt.figure(figsize=(10, 8))
    plt.subplot(2, 2, 1)
    plt.title('Original Image')
    plt.imshow(image)
    
    # Increase brightness (simple tensor operation)
    brightened = tf.clip_by_value(image * 1.5, 0.0, 1.0)
    plt.subplot(2, 2, 2)
    plt.title('Brightened Image')
    plt.imshow(brightened)
    
    # Convert to grayscale using weighted sum
    rgb_weights = tf.constant([0.299, 0.587, 0.114])
    grayscale = tf.reduce_sum(image * rgb_weights, axis=2, keepdims=True)
    grayscale = tf.repeat(grayscale, 3, axis=2)  # Make it 3 channels again for display
    plt.subplot(2, 2, 3)
    plt.title('Grayscale')
    plt.imshow(grayscale)
    
    # Flip the image horizontally
    flipped = tf.image.flip_left_right(image)
    plt.subplot(2, 2, 4)
    plt.title('Flipped Horizontally')
    plt.imshow(flipped)
    
    plt.tight_layout()
    plt.show()
    
except Exception as e:
    print("Couldn't load the image. Error:", e)
    
    # Create a simple synthetic image instead
    print("Creating a synthetic image instead")
    synthetic_image = tf.random.uniform(shape=(100, 100, 3))
    
    plt.figure(figsize=(5, 5))
    plt.imshow(synthetic_image)
    plt.title('Synthetic Random Image')
    plt.show()
    
    print("Synthetic image tensor shape:", synthetic_image.shape)
    print("Data type:", synthetic_image.dtype)

Practical Example: Using Tensors in a Neural Network

Here's a simple example of how tensors are used in neural networks with TensorFlow:

python

# Creating a simple neural network layer
inputs = tf.constant([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])
print("Input tensor shape:", inputs.shape)  # (2, 3) - 2 examples with 3 features each

# Define weights and biases
weights = tf.Variable(tf.random.normal([3, 2]))  # 3 input features, 2 output features
biases = tf.Variable(tf.zeros([2]))  # 2 output features

print("\nWeights tensor shape:", weights.shape)
print("Biases tensor shape:", biases.shape)

# Forward pass calculation
outputs = tf.matmul(inputs, weights) + biases
print("\nOutput tensor (after forward pass):\n", outputs)
print("Output tensor shape:", outputs.shape)  # (2, 2) - 2 examples with 2 outputs each

# Apply an activation function
activated_outputs = tf.nn.relu(outputs)  # ReLU activation: max(0, x)
print("\nActivated outputs (after ReLU):\n", activated_outputs)

Output (will vary due to random initialization):

Input tensor shape: (2, 3)

Weights tensor shape: (3, 2)
Biases tensor shape: (2,)

Output tensor (after forward pass):
 tf.Tensor(
[[ 0.14675189 -0.5081452 ]
 [ 0.46359882 -1.0240135 ]], shape=(2, 2), dtype=float32)
Output tensor shape: (2, 2)

Activated outputs (after ReLU):
 tf.Tensor(
[[0.14675189 0.        ]
 [0.46359882 0.        ]], shape=(2, 2), dtype=float32)

Summary

In this tutorial, you've learned about tensors, the fundamental building blocks of TensorFlow:

• Tensors are multi-dimensional arrays with uniform data types
• You can create tensors of various shapes and types using tf.constant, tf.zeros, tf.ones, and other functions
• Tensors have attributes like shape, rank, and data type
• You can perform mathematical operations on tensors just like with regular numbers
• Tensors can be reshaped, sliced, and transposed to fit your needs
• We explored practical examples of using tensors for image processing and neural networks

Understanding tensors is critical for working effectively with TensorFlow and deep learning in general. As you continue your journey, you'll find that most of the operations in TensorFlow are built around manipulating tensors efficiently.

Additional Resources

1. TensorFlow Official Documentation on Tensors
2. TensorFlow Math Operations
3. Introduction to Tensors for Deep Learning (TensorFlow Blog)

Exercises

1. Create a 4x4 identity matrix using TensorFlow functions.
2. Write code to calculate the mean and standard deviation of values in a tensor.
3. Create a 3D tensor with random values and practice slicing it in different ways.
4. Create two tensors and concatenate them along different axes.
5. Load a small dataset (like MNIST) and explore how the image data is represented as tensors.