Skip to main content

Authentication Methods

Introduction​

Authentication is a critical component in Grafana Loki's multi-tenant architecture. It ensures that users and services can securely identify themselves when accessing Loki resources, while maintaining proper isolation between different tenants. In this guide, we'll explore the various authentication methods available in Loki, how they work, and how to implement them effectively in a multi-tenant environment.

Authentication Basics in Loki​

Loki's authentication system is designed to verify the identity of clients (users, applications, or services) attempting to interact with the system. In a multi-tenant setup, authentication serves as the first line of defense in ensuring tenant isolation and data security.

Why Authentication Matters in Multi-Tenancy​

In a multi-tenant environment, authentication serves several crucial purposes:

  1. Tenant Isolation: Ensures that tenants can only access their own data
  2. Resource Control: Enables proper allocation of resources per tenant
  3. Audit Trail: Provides trackable records of who accessed what
  4. Security Boundaries: Prevents unauthorized access across tenant boundaries

Supported Authentication Methods​

Loki supports several authentication methods that can be used in a multi-tenant environment. Let's explore each one in detail.

1. HTTP Basic Authentication​

The simplest form of authentication supported by Loki involves using HTTP Basic Authentication headers.

How It Works​

The client sends an HTTP request with an Authorization header containing the word Basic followed by a base64-encoded string of username:password.

bash
curl -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" http://loki:3100/loki/api/v1/labels

Configuration Example​

In your Loki configuration file:

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: basic
  basic:
    username: username
    password: password

Pros and Cons​

Pros:

  • Simple to implement
  • Widely supported by tools and clients

Cons:

  • Credentials sent with every request
  • Limited security (base64 encoding is not encryption)
  • Not suitable for production environments without HTTPS

2. Bearer Token Authentication​

A more secure alternative, Bearer Token authentication uses tokens instead of username/password combinations.

How It Works​

The client includes an Authorization header with a Bearer token in requests:

bash
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9..." http://loki:3100/loki/api/v1/labels

Configuration Example​

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: token
  token:
    backend: file
    file:
      path: /etc/loki/tokens.yaml

Where tokens.yaml might look like:

yaml
- tenant: tenant1
  token: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
- tenant: tenant2
  token: dGhpcyBpcyBhIHRva2VuIGZvciB0ZW5hbnQy...

Pros and Cons​

Pros:

  • More secure than basic authentication
  • Can include expiration and other claims
  • Suitable for service-to-service communication

Cons:

  • Token management can be complex
  • Requires secure storage of tokens

3. JWT Authentication​

JSON Web Tokens (JWT) provide a self-contained way to transmit information securely between parties.

How It Works​

JWT tokens consist of three parts: a header, a payload, and a signature. The payload can contain claims about the tenant and permissions.

bash
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnRfaWQiOiJ0ZW5hbnQxIn0.signature" http://loki:3100/loki/api/v1/labels

Configuration Example​

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: jwt
  jwt:
    secret: your-secret-key
    tenant_claim: tenant_id

JWT Token Generation Example​

Here's how you can generate a JWT token for testing:

javascript
const jwt = require('jsonwebtoken');

const token = jwt.sign(
  { tenant_id: 'tenant1' },
  'your-secret-key',
  { expiresIn: '1h' }
);

console.log(token);
// Output: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ0ZW5hbnRfaWQiOiJ0ZW5hbnQxIiwiZXhwIjoxNjE3MjkwNjA1fQ.signature

Pros and Cons​

Pros:

  • Stateless authentication
  • Can include rich tenant information
  • Flexible and widely adopted

Cons:

  • Requires proper key management
  • Token size can be larger than simple bearer tokens

4. OIDC (OpenID Connect) Authentication​

OpenID Connect extends OAuth 2.0 to add an identity layer, making it ideal for multi-tenant environments with user-based authentication.

How It Works​

OIDC authentication flow:

Configuration Example​

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: oidc
  oidc:
    client_id: loki-client
    client_secret: your-client-secret
    issuer: https://your-identity-provider/
    tenant_claim: groups

Pros and Cons​

Pros:

  • Complete identity management solution
  • Well-suited for user authentication
  • Integrates with existing identity providers

Cons:

  • More complex to set up
  • Requires external identity provider

5. X.509 Client Certificate Authentication​

Using TLS client certificates for mutual authentication.

How It Works​

The client presents a certificate during the TLS handshake, which Loki validates against trusted certificate authorities.

Configuration Example​

yaml
auth_enabled: true
server:
  http_listen_port: 3100
  grpc_listen_port: 9095
  http_server_read_timeout: 10s
  http_server_write_timeout: 10s
  grpc_server_max_recv_msg_size: 4194304
  grpc_server_max_send_msg_size: 4194304
  tls_config:
    cert_file: /path/to/server.crt
    key_file: /path/to/server.key
    client_auth_type: RequireAndVerifyClientCert
    client_ca_file: /path/to/ca.crt

auth:
  type: certificate
  certificate:
    tenant_field: O  # Organization field in the certificate

Pros and Cons​

Pros:

  • Very secure authentication method
  • Certificates can include tenant information
  • No need to send credentials with each request

Cons:

  • Complex certificate management
  • Not as flexible for dynamic environments

Multi-Factor Authentication​

For critical deployments, combining multiple authentication methods provides enhanced security.

Configuration Example​

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: multi
  multi:
    - type: jwt
      jwt:
        secret: your-jwt-secret
        tenant_claim: tenant_id
    - type: basic
      basic:
        username: emergency-user
        password: complex-password

Choosing the Right Authentication Method​

When deciding which authentication method to use, consider:

  1. Security requirements: How sensitive is your data?
  2. Existing infrastructure: What authentication systems do you already use?
  3. User experience: Who will be authenticating with Loki?
  4. Operational complexity: Can you manage the chosen method effectively?

This decision matrix can help you choose:

Authentication MethodSecurity LevelImplementation ComplexitySuitable For
HTTP BasicLowLowDevelopment, simple deployments
Bearer TokenMediumMediumService-to-service, simple production
JWTMedium-HighMediumMulti-service environments
OIDCHighHighUser-facing applications
X.509 CertificatesVery HighVery HighHighly secure environments

Implementing Tenant-Aware Authentication​

In a multi-tenant environment, your authentication method should be integrated with tenant identification.

Example: JWT with Tenant Claim​

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: jwt
  jwt:
    secret: your-secret-key
    tenant_claim: tenant_id

With this configuration, Loki extracts the tenant ID from the tenant_id claim in the JWT payload.

Example: Using HTTP Headers​

For clients that don't support JWT, you can use the X-Scope-OrgID header:

bash
curl -H "Authorization: Bearer your-token" -H "X-Scope-OrgID: tenant1" http://loki:3100/loki/api/v1/labels

Testing Authentication​

To test your authentication configuration, you can use the following command:

bash
curl -v -H "Authorization: Bearer your-token" -H "X-Scope-OrgID: tenant1" http://loki:3100/loki/api/v1/labels

A successful authentication will return HTTP status 200 with your tenant's label data. Authentication failures typically return HTTP status 401 (Unauthorized) or 403 (Forbidden).

Authentication in Real-World Scenarios​

Scenario 1: Multi-Team Development Environment​

In a development setup with multiple teams sharing a Loki instance:

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: token
  token:
    backend: file
    file:
      path: /etc/loki/tokens.yaml

With tokens.yaml:

yaml
- tenant: team-a
  token: team-a-token
- tenant: team-b
  token: team-b-token

Each team uses their assigned token to push and query logs, maintaining isolation between teams.

Scenario 2: Production Microservices​

For a production environment with multiple microservices:

yaml
auth_enabled: true
server:
  http_listen_port: 3100

auth:
  type: jwt
  jwt:
    secret: production-secret
    tenant_claim: service_name

Each service generates a JWT with its service name as the tenant identifier, allowing for service-level isolation.

Security Best Practices​

When implementing authentication in Loki:

  1. Always use TLS/HTTPS to encrypt traffic
  2. Rotate secrets and tokens regularly
  3. Use the principle of least privilege when assigning permissions
  4. Implement rate limiting to prevent abuse
  5. Monitor authentication failures to detect potential attacks
  6. Back up authentication configurations securely

Troubleshooting Authentication Issues​

Common Problems and Solutions​

  1. 401 Unauthorized Errors

    • Check that your token or credentials are correct
    • Verify the token hasn't expired
    • Ensure the authentication header format is correct

  2. 403 Forbidden Errors

    • The tenant ID might be missing or incorrect
    • Check that the authenticated tenant has the necessary permissions

  3. Authentication Works But No Data Visible

    • Verify you're querying with the same tenant ID used during ingestion
    • Check that the tenant claim is correctly configured

Summary​

Authentication is a critical component of Loki's multi-tenant architecture. By understanding the available authentication methods and their appropriate use cases, you can implement a secure, scalable logging solution that maintains proper isolation between tenants.

In this guide, we've explored:

  • The importance of authentication in multi-tenant environments
  • Different authentication methods supported by Loki
  • How to configure each method
  • Real-world implementation scenarios
  • Security best practices and troubleshooting

Remember that the right authentication method depends on your specific requirements, existing infrastructure, and security needs.

Additional Resources​

Exercises​

  1. Configure Loki with basic authentication and test accessing labels API
  2. Generate a JWT token with tenant information and configure Loki to use it
  3. Set up a multi-tenant environment with different authentication methods for different clients
  4. Implement a rotation strategy for your authentication tokens or secrets

If you spot any mistakes on this website, please let me know at [email protected]. I’d greatly appreciate your feedback! :)