MongoDB mongoexport

Introduction

When working with MongoDB, there are often scenarios where you need to export data from your collections to share with other systems, create backups, or migrate to different environments. The mongoexport utility is a command-line tool provided by MongoDB that allows you to export data from MongoDB collections into JSON or CSV formats.

Unlike other MongoDB backup methods like mongodump (which creates BSON files), mongoexport generates human-readable files that can be easily viewed, edited, or imported into other systems, making it ideal for data interchange scenarios.

Prerequisites

Before using mongoexport, ensure you have:

• MongoDB installed (the utility comes with MongoDB installation)
• Access to a MongoDB instance
• Appropriate permissions to read from the collections you want to export

Basic Syntax

The basic syntax for mongoexport is:

bash

mongoexport --uri=<connection_string> --collection=<collection_name> --out=<output_file>

Key Parameters

Here are the most commonly used parameters for mongoexport:

Parameter	Description
--uri	MongoDB connection string
--host	MongoDB server hostname or IP address
--port	MongoDB server port
--db	Database name to export from
--collection	Collection name to export
--out	Output file path
--fields	Comma-separated list of field names to export
--query	JSON query to filter documents for export
--type	Output format: json (default) or csv
--jsonArray	Export data as a JSON array instead of one JSON document per line

Basic Examples

Exporting an Entire Collection to JSON

Let's export all documents from a users collection in the myapp database to a JSON file:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --out=users.json

Output:

2023-11-14T10:15:32.654+0100    connected to: mongodb://localhost:27017/myapp
2023-11-14T10:15:32.789+0100    exported 1250 documents

The output file (users.json) will contain one JSON document per line:

json

{"_id":{"$oid":"5f8d76a3e6ab2d1234567890"},"name":"John Doe","email":"[email protected]","age":30}
{"_id":{"$oid":"5f8d76b7e6ab2d1234567891"},"name":"Jane Smith","email":"[email protected]","age":28}

Exporting as a JSON Array

If you prefer having your data as a proper JSON array, use the --jsonArray flag:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --out=users_array.json --jsonArray

This produces a valid JSON array in users_array.json:

json

[
  {"_id":{"$oid":"5f8d76a3e6ab2d1234567890"},"name":"John Doe","email":"[email protected]","age":30},
  {"_id":{"$oid":"5f8d76b7e6ab2d1234567891"},"name":"Jane Smith","email":"[email protected]","age":28}
]

Exporting to CSV Format

To export data to CSV format, use the --type=csv parameter along with --fields to specify which fields to include:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --out=users.csv --type=csv --fields=name,email,age

This creates a CSV file (users.csv):

name,email,age
John Doe,[email protected],30
Jane Smith,[email protected],28

Advanced Usage

Filtering Documents with Query

You can use the --query parameter to export only documents that match a specific condition:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --out=adults.json --query='{"age":{"$gte":18}}' 

This exports only users aged 18 or older.

Projecting Specific Fields

If you only need certain fields, use the --fields parameter:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --out=user_contacts.json --fields=name,email,phone

Handling Authentication

For databases that require authentication:

bash

mongoexport --uri="mongodb://username:password@localhost:27017/myapp" --collection=users --out=users.json

Or using separate parameters:

bash

mongoexport --host localhost --port 27017 --db myapp --collection users --username admin --password "password123" --authenticationDatabase admin --out=users.json

Exporting Date Ranges

To export documents created within a specific date range:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=transactions --query='{"createdAt":{"$gte":{"$date":"2023-01-01T00:00:00Z"},"$lt":{"$date":"2023-02-01T00:00:00Z"}}}' --out=jan_transactions.json

Real-world Examples

Example 1: Data Migration Between Environments

When migrating data from development to staging environments:

bash

# Export from development
mongoexport --uri="mongodb://localhost:27017/myapp_dev" --collection=products --out=products.json

# Import to staging 
mongoimport --uri="mongodb://staging-server:27017/myapp_staging" --collection=products --file=products.json

Example 2: Creating a Dataset for Analytics

When preparing data for analysis in another tool:

bash

mongoexport --uri="mongodb://localhost:27017/myapp" --collection=orders --out=orders_for_analysis.csv --type=csv --fields=customer_id,product_id,amount,date,status

Example 3: Regular Data Backup Script

Create a shell script for automated daily backups:

bash

#!/bin/bash
# File: backup_mongodb.sh

# Set variables
DATABASE="myapp"
BACKUP_DIR="/backups/mongodb"
DATE=$(date +%Y-%m-%d)

# Create backup directory if it doesn't exist
mkdir -p $BACKUP_DIR

# Export collections
collections=("users" "products" "orders" "settings")

for collection in "${collections[@]}"
do
  mongoexport --uri="mongodb://localhost:27017/$DATABASE" \
    --collection=$collection \
    --out=$BACKUP_DIR/${collection}_${DATE}.json
done

# Compress all files from today
cd $BACKUP_DIR
tar -czf mongodb_export_${DATE}.tar.gz *_${DATE}.json

# Remove individual JSON files
rm *_${DATE}.json

echo "Backup completed: $BACKUP_DIR/mongodb_export_${DATE}.tar.gz"

Best Practices

1. Set appropriate permissions on the output files to protect potentially sensitive data.
2. Use query filters to export only the necessary data, especially for large collections.
3. Include timestamps in output filenames for tracking when exports were created.
4. Be cautious with large exports as they can impact database performance.
5. Consider using compression for large exports to save disk space.
6. Check for errors in the export process by examining the command output.

Common Issues and Troubleshooting

Large Data Sets

When exporting very large collections, you might encounter memory issues or timeouts. You can:

1. Use query filters to export in smaller chunks:

bash

# Export users in batches by ID range
mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --query='{"_id":{"$lt":{"$oid":"5f8d76a3e6ab2d0000000100"}}}' --out=users_batch1.json
mongoexport --uri="mongodb://localhost:27017/myapp" --collection=users --query='{"_id":{"$gte":{"$oid":"5f8d76a3e6ab2d0000000100"},"$lt":{"$oid":"5f8d76a3e6ab2d0000000200"}}}' --out=users_batch2.json

2. Increase the timeout settings:

bash

mongoexport --uri="mongodb://localhost:27017/myapp?socketTimeoutMS=120000" --collection=large_data --out=large_data.json

Nested Objects and Arrays

CSV exports don't handle nested objects and arrays well. For complex documents, JSON format is usually better:

bash

# This might not work as expected with CSV
mongoexport --uri="mongodb://localhost:27017/myapp" --collection=complex_data --type=csv --fields=name,address.city,tags --out=complex_data.csv

# Better to use JSON format
mongoexport --uri="mongodb://localhost:27017/myapp" --collection=complex_data --out=complex_data.json

Limitations

• mongoexport is not suitable for full database backups (use mongodump instead)
• CSV exports don't handle complex BSON types well
• Large exports can be resource-intensive
• Not ideal for maintaining relationships between collections

Summary

mongoexport is a powerful utility for exporting MongoDB data to JSON or CSV formats. It's particularly useful for:

• Creating human-readable exports of MongoDB data
• Sharing data with non-MongoDB systems
• Creating simple backups of individual collections
• Migrating specific data between environments
• Generating datasets for analysis in other tools

By understanding the various parameters and usage patterns of mongoexport, you can effectively leverage it in your data migration and integration workflows.

Practice Exercises

1. Export all documents from a users collection where status is "active" as a JSON array.
2. Create a CSV export of products collection with only the fields name, price, and category.
3. Write a script that exports all collections in a database to separate JSON files.
4. Export documents created in the last 7 days from an orders collection.
5. Export a collection with proper authentication and save it with a timestamp in the filename.

Additional Resources

• MongoDB Official Documentation on mongoexport
• MongoDB Database Tools
• mongoimport Documentation (companion tool for importing data)

Remember that while mongoexport is excellent for specific use cases, for comprehensive backups with proper BSON type preservation, consider using mongodump and mongorestore instead.