Skip to main content

Redis Lua Scripting

Introduction​

Redis Lua Scripting is a powerful feature that allows you to execute Lua scripts within Redis. This capability transforms Redis from a simple key-value store into a more versatile data processing engine, enabling you to perform complex operations atomically and efficiently.

In this tutorial, we'll explore how Redis Lua scripts work, why they're useful, and how to implement them in your applications. By the end, you'll be able to leverage Lua scripts to enhance your Redis operations and solve real-world problems.

Why Use Lua Scripting in Redis?​

Before diving into the implementation details, let's understand the benefits of using Lua scripts in Redis:

  1. Atomicity: Lua scripts execute as a single atomic operation, preventing race conditions in multi-step processes.
  2. Reduced Network Overhead: Instead of sending multiple commands to Redis, you can send a single script that runs on the server.
  3. Simplified Complex Operations: Some operations that would require multiple Redis commands can be simplified into a single Lua script.
  4. Performance: Scripts execute directly on the Redis server, reducing latency for complex operations.

Getting Started with Redis Lua​

Basic Syntax​

Redis uses the EVAL command to execute Lua scripts. The basic syntax is:

EVAL script numkeys key [key ...] arg [arg ...]

Where:

  • script is your Lua code as a string
  • numkeys specifies how many of the following arguments are keys
  • key [key ...] are the Redis keys your script will access
  • arg [arg ...] are additional arguments to pass to the script

Your First Lua Script​

Let's create a simple Lua script that increments a value and returns the new value:

EVAL "return redis.call('INCR', KEYS[1])" 1 mycounter

Breaking this down:

  • return redis.call('INCR', KEYS[1]) is our Lua script
  • 1 indicates we're passing 1 key
  • mycounter is the key we're operating on

Input:

> EVAL "return redis.call('INCR', KEYS[1])" 1 mycounter

Output:

(integer) 1

If we run it again:

Input:

> EVAL "return redis.call('INCR', KEYS[1])" 1 mycounter

Output:

(integer) 2

Passing Arguments​

You can also pass additional arguments to your script, which are accessible via the ARGV table:

EVAL "return redis.call('SET', KEYS[1], ARGV[1])" 1 mykey "Hello, Redis Lua!"

Input:

> EVAL "return redis.call('SET', KEYS[1], ARGV[1])" 1 mykey "Hello, Redis Lua!"
> GET mykey

Output:

OK
"Hello, Redis Lua!"

Script Execution Context​

When executing Lua scripts in Redis:

  1. Scripts run in a sandboxed environment
  2. Scripts are executed atomically - no other Redis operations will interrupt your script
  3. All Redis commands are available through the redis.call() function
  4. You can access keys and arguments through the KEYS and ARGV tables

Advanced Redis Lua Techniques​

Script Reusability with SCRIPT LOAD and EVALSHA​

Instead of sending your script each time, you can load it once and call it by its SHA1 hash:

SCRIPT LOAD "return redis.call('GET', KEYS[1])"

This returns a SHA1 hash that you can use with EVALSHA:

Input:

> SCRIPT LOAD "return redis.call('GET', KEYS[1])"

Output:

"a42059b356c875f0717db19a51f6aaca9ae659ea"

Now use it:

Input:

> SET mykey "testing EVALSHA"
> EVALSHA "a42059b356c875f0717db19a51f6aaca9ae659ea" 1 mykey

Output:

OK
"testing EVALSHA"

This approach is more efficient for scripts you execute frequently.

Error Handling​

Redis Lua scripts provide two ways to call Redis commands:

  1. redis.call(): Raises an error if the Redis command fails
  2. redis.pcall(): Returns the error instead of raising it

Example using pcall for safer execution:

EVAL "local result = redis.pcall('GET', KEYS[1]); if type(result) == 'table' and result.err then return 'Error: ' .. result.err else return result end" 1 nonexistent_key

Working with Lua Data Structures​

Lua provides powerful data structure manipulation capabilities:

lua
EVAL "
local ids = {}
for i=1,5 do
  table.insert(ids, redis.call('INCR', KEYS[1]))
end
return ids
" 1 sequence_counter

Input:

> EVAL "local ids = {}; for i=1,5 do table.insert(ids, redis.call('INCR', KEYS[1])); end; return ids" 1 sequence_counter

Output:

1) (integer) 1
2) (integer) 2
3) (integer) 3
4) (integer) 4
5) (integer) 5

Real-World Examples​

Example 1: Atomic Counter with Expiration​

This script increments a counter and sets an expiration time if the counter didn't exist before:

lua
EVAL "
local current = redis.call('INCR', KEYS[1])
if current == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return current
" 1 visit_counter 3600

Input:

> EVAL "local current = redis.call('INCR', KEYS[1]); if current == 1 then redis.call('EXPIRE', KEYS[1], ARGV[1]); end; return current" 1 visit_counter 3600
> TTL visit_counter

Output:

(integer) 1
(integer) 3600

Example 2: Implementing a Rate Limiter​

A common use case for Lua scripting is implementing a rate limiter:

lua
EVAL "
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])

local current = redis.call('INCR', key)
if current == 1 then
  redis.call('EXPIRE', key, window)
end

if current > limit then
  return 0
else
  return 1
end
" 1 rate_limit:user123 5 60

This script:

  1. Increments a counter for the specified key (e.g., user ID)
  2. Sets a 60-second expiration when the counter is first created
  3. Returns 0 if the user has exceeded 5 requests in 60 seconds, or 1 if they're within limits

Input:

> EVAL "local key = KEYS[1]; local limit = tonumber(ARGV[1]); local window = tonumber(ARGV[2]); local current = redis.call('INCR', key); if current == 1 then redis.call('EXPIRE', key, window); end; if current > limit then return 0 else return 1 end" 1 rate_limit:user123 5 60

Output:

(integer) 1

After running it 6 times:

(integer) 0

Example 3: Implementing a Simple Queue​

lua
EVAL "
local push = function(list, item)
  redis.call('RPUSH', list, item)
  return redis.call('LLEN', list)
end

local queue = KEYS[1]
local item = ARGV[1]
return push(queue, item)
" 1 my_queue "job data"

Input:

> EVAL "local push = function(list, item) redis.call('RPUSH', list, item); return redis.call('LLEN', list); end; local queue = KEYS[1]; local item = ARGV[1]; return push(queue, item)" 1 my_queue "job data"

Output:

(integer) 1

Implementation in Different Languages​

Node.js (with redis client)​

javascript
const redis = require('redis');
const client = redis.createClient();

// Simple counter script
const counterScript = `
local current = redis.call('INCR', KEYS[1])
if current == 1 then
  redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return current
`;

// Execute the script
client.eval(counterScript, 1, 'visitor_count', 3600, (err, result) => {
  if (err) {
    console.error('Error executing script:', err);
    return;
  }
  console.log('Current count:', result);
});

Python (with redis-py)​

python
import redis

r = redis.Redis()

# Rate limiter script
rate_limiter = """
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local window = tonumber(ARGV[2])

local current = redis.call('INCR', key)
if current == 1 then
  redis.call('EXPIRE', key, window)
end

if current > limit then
  return 0
else
  return 1
end
"""

# Load the script once
rate_limit_sha = r.script_load(rate_limiter)

# Check if user is rate limited
user_id = "user123"
allowed = r.evalsha(rate_limit_sha, 1, f"rate_limit:{user_id}", 5, 60)

if allowed:
    print("Request allowed")
else:
    print("Rate limit exceeded")

Best Practices​

When working with Redis Lua scripts, keep these best practices in mind:

  1. Keep scripts simple and focused: Complex scripts can block Redis longer.
  2. Use SCRIPT LOAD and EVALSHA for frequently used scripts to reduce bandwidth.
  3. Limit script execution time: Avoid infinite loops or heavy computations.
  4. Use KEYS for read-only operations and ARGV for data: This makes your intention clearer.
  5. Document your scripts well: Scripts can become complex and hard to maintain.
  6. Use pcall for error handling: This prevents script aborts when Redis commands fail.
  7. Consider using LUA_REPLICATE_COMMANDS: For scripts that are deterministic and can be replicated.

Debugging Lua Scripts​

Debugging Lua scripts in Redis can be challenging. Here are some tips:

  1. Test scripts locally: Develop and test your script logic first before running in Redis.
  2. Use print for debugging: You can use redis.log(redis.LOG_NOTICE, "Debug message") in your scripts.
  3. Break complex scripts into smaller parts: This makes them easier to debug.

For example:

lua
EVAL "
redis.log(redis.LOG_NOTICE, 'Processing key: ' .. KEYS[1])
local value = redis.call('GET', KEYS[1])
redis.log(redis.LOG_NOTICE, 'Value: ' .. tostring(value))
return value
" 1 mykey

Common Pitfalls​

Watch out for these common issues when working with Redis Lua scripts:

  1. Script complexity: Scripts run atomically and block Redis during execution.
  2. Memory usage: Large scripts can consume significant memory.
  3. Script determinism: Non-deterministic operations (like RANDOMKEY) may behave differently on replicas.
  4. Execution time limits: Redis has a lua-time-limit configuration (default: 5 seconds).

Summary​

Redis Lua scripting is a powerful feature that allows you to extend Redis functionality by running atomic, server-side operations. By leveraging Lua scripts, you can:

  • Reduce network overhead
  • Ensure atomic execution of complex operations
  • Implement custom logic within Redis
  • Improve performance for multi-step operations

In this tutorial, we've covered the basics of Redis Lua scripting, advanced techniques, and real-world examples. You should now be equipped to start implementing your own Lua scripts in Redis to solve your specific use cases.

Exercises​

  1. Write a Lua script that implements a simple leaderboard, adding a score for a user and returning their rank.
  2. Create a script that implements a distributed lock mechanism with automatic expiration.
  3. Implement a "get and set if not exists" operation using Lua.
  4. Write a script that manages a sliding window rate limiter with a specified time window.
  5. Create a script that atomically moves an item from one list to another.

Additional Resources​


If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)