Skip to content

ZainTechnologiesLTD/Multi-Node-Mail-Server-Setup

BranchesTags

Repository files navigation

Carbonio CE 25.9.0 - 7 Node Full Standard Deployment

Complete Infrastructure as Code Setup: Kubernetes + Terraform + Docker + Ansible + Prometheus

Overview

This repository contains a complete Infrastructure as Code (IaC) solution for deploying Carbonio Community Edition 25.9.0 in a 7-node Full Standard scenario on Ubuntu Server 24.04.3 with:

  • Kubernetes orchestration for container management
  • Terraform for Infrastructure as Code and resource provisioning
  • Docker containers for each node (Ubuntu 24.04.3 base)
  • Ansible for configuration management and deployment
  • Prometheus for comprehensive monitoring and alerting

Architecture Overview

Node Distribution

Node Component Services
Node 1 Core (Mesh & Directory) Mesh, Directory Service, PostgreSQL Database, Service Discover, Syslog, Prometheus
Node 2 MTA Mail Transfer Agent, Anti-Virus, Anti-Spam
Node 3 Proxy Reverse Proxy, Load Balancer, Webmail
Node 4 Mailstore Mailstore, Provisioning, Tasks
Node 5 Collaboration Files, Chats, WorkStream
Node 6 Documents Docs, Editor, Preview
Node 7 Video Video Server, Video Recording

System Requirements

  • Host OS: Ubuntu Server 24.04.3
  • Storage: 500 GB
  • RAM: 19 GB total (allocated across nodes)
  • Network: All nodes must be able to reach each other via SSH/networking

Prerequisites

On Control Node (Your Machine)

  1. Ansible (≥ 2.9)

    sudo apt-get install ansible

  2. Terraform (≥ 1.0)

    curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo apt-key add -
sudo apt-add-repository "deb [arch=amd64] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
sudo apt-get update && sudo apt-get install terraform

  3. Docker (≥ 20.10)

    curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh

  4. kubectl (≥ 1.24)

    curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl

  5. Git

    sudo apt-get install git

On Target Nodes

  • Ubuntu Server 24.04.3 or later
  • SSH access as root
  • Carbonio repository configured
  • Network connectivity between all nodes

Directory Structure

carbonio-kubernetes-terraform/
├── README.md                          # This file
├── LICENSE
├── ARCHITECTURE.md                    # Detailed architecture documentation
├── INSTALLATION_GUIDE.md              # Step-by-step installation guide
├── TROUBLESHOOTING.md                 # Common issues and solutions
│
├── docker/                            # Docker configuration
│   ├── Dockerfile.ubuntu              # Base Ubuntu 24.04.3 Dockerfile
│   ├── docker-compose.yml             # Docker Compose for 7-node setup
│   ├── docker-compose.override.yml    # Development overrides
│   ├── .dockerignore
│   └── scripts/
│       └── setup-node.sh              # Node initialization script
│
├── kubernetes/                        # Kubernetes manifests
│   ├── namespace.yaml                 # Carbonio namespace
│   ├── nodes/
│   │   ├── node1-core-deployment.yaml
│   │   ├── node2-mta-deployment.yaml
│   │   ├── node3-proxy-deployment.yaml
│   │   ├── node4-mailstore-deployment.yaml
│   │   ├── node5-collab-deployment.yaml
│   │   ├── node6-docs-deployment.yaml
│   │   └── node7-video-deployment.yaml
│   ├── services/
│   │   └── carbonio-services.yaml     # Service definitions
│   ├── configmaps/
│   │   └── carbonio-config.yaml       # Configuration maps
│   ├── pvs/
│   │   └── persistent-volumes.yaml    # Persistent volumes
│   ├── rbac/
│   │   └── rbac.yaml                  # RBAC policies
│   └── helm/
│       ├── Chart.yaml                 # Helm chart
│       ├── values.yaml                # Helm values
│       └── templates/                 # Helm templates
│
├── terraform/                         # Terraform infrastructure
│   ├── main.tf                        # Main Terraform configuration
│   ├── variables.tf                   # Input variables
│   ├── outputs.tf                     # Output values
│   ├── provider.tf                    # Provider configuration
│   ├── networking.tf                  # Network resources
│   ├── kubernetes-cluster.tf          # K8s cluster definition
│   ├── docker-infrastructure.tf       # Docker nodes
│   ├── storage.tf                     # Storage configuration
│   ├── monitoring.tf                  # Monitoring resources
│   ├── terraform.tfvars               # Environment-specific variables
│   └── modules/                       # Reusable modules
│       ├── docker-node/
│       ├── kubernetes-cluster/
│       └── monitoring/
│
├── ansible/                           # Ansible playbooks
│   ├── inventory/
│   │   ├── carbonio-inventory         # Inventory file (hosts)
│   │   └── group_vars/
│   │       ├── all.yml                # Global variables
│   │       ├── core_nodes.yml
│   │       ├── mta_nodes.yml
│   │       ├── proxy_nodes.yml
│   │       ├── mailstore_nodes.yml
│   │       ├── collab_nodes.yml
│   │       ├── docs_nodes.yml
│   │       └── video_nodes.yml
│   ├── roles/                         # Ansible roles
│   │   ├── common/
│   │   ├── carbonio-base/
│   │   ├── core/
│   │   ├── mta/
│   │   ├── proxy/
│   │   ├── mailstore/
│   │   ├── collab/
│   │   ├── docs/
│   │   └── video/
│   ├── site.yml                       # Main playbook
│   ├── deploy-all.yml                 # Deploy all nodes
│   ├── deploy-core.yml                # Deploy core node
│   ├── ansible.cfg                    # Ansible configuration
│   └── roles-galaxy/
│       └── requirements.yml           # Galaxy role requirements
│
├── prometheus/                        # Prometheus monitoring
│   ├── prometheus.yml                 # Prometheus configuration
│   ├── alerts/
│   │   ├── carbonio-alerts.yaml       # Alert rules
│   │   ├── prometheus-alerts.yaml
│   │   └── kubernetes-alerts.yaml
│   ├── grafana/
│   │   ├── dashboards/
│   │   │   ├── carbonio-overview.json
│   │   │   ├── node-exporter.json
│   │   │   ├── postgres.json
│   │   │   └── docker-stats.json
│   │   ├── datasources/
│   │   │   └── prometheus.yaml
│   │   └── provisioning/
│   ├── exporters/
│   │   ├── node-exporter.yml
│   │   ├── postgres-exporter.yml
│   │   └── docker-exporter.yml
│   └── alertmanager.yml               # AlertManager configuration
│
├── scripts/                           # Utility scripts
│   ├── setup.sh                       # Initial setup script
│   ├── deploy.sh                      # Deployment orchestration script
│   ├── validate.sh                    # Validation script
│   ├── health-check.sh                # Health check script
│   └── cleanup.sh                     # Cleanup script
│
└── docs/                              # Additional documentation
    ├── ARCHITECTURE.md
    ├── INSTALLATION_GUIDE.md
    ├── TROUBLESHOOTING.md
    ├── API_REFERENCE.md
    └── MONITORING.md

Quick Start

1. Clone and Setup

git clone https://github.com/yourusername/carbonio-kubernetes-terraform.git
cd carbonio-kubernetes-terraform
chmod +x scripts/*.sh

2. Configure Terraform Variables

Edit terraform/terraform.tfvars:

project_name           = "carbonio"
environment            = "production"
ubuntu_version         = "24.04.3"
docker_network_subnet  = "172.20.0.0/16"
kubernetes_version     = "1.28"
total_ram_gb           = 19
total_storage_gb       = 500

3. Configure Ansible Inventory

Edit ansible/inventory/carbonio-inventory:

[postgresServers]
carbonio-node1 ansible_host=172.20.0.10 ansible_user=root

[masterDirectoryServers]
carbonio-node1

[serviceDiscoverServers]
carbonio-node1

[mtaServers]
carbonio-node2 ansible_host=172.20.0.20 ansible_user=root

[proxyServers]
carbonio-node3 ansible_host=172.20.0.30 ansible_user=root webmailHostname=mail.example.com

[applicationServers]
carbonio-node4 ansible_host=172.20.0.40 ansible_user=root

[filesServers]
carbonio-node5 ansible_host=172.20.0.50 ansible_user=root

[docsServers]
carbonio-node6 ansible_host=172.20.0.60 ansible_user=root

[taskServers]
carbonio-node4

[previewServers]
carbonio-node6

[videoServers]
carbonio-node7 ansible_host=172.20.0.70 ansible_user=root

[prometheusServers]
carbonio-node1

[syslogServer]
carbonio-node1

[workStreamServers]
carbonio-node5

4. Deploy Infrastructure

# Deploy Docker containers
docker-compose -f docker/docker-compose.yml up -d

# Apply Kubernetes manifests (if using K8s)
kubectl apply -f kubernetes/

# Or use Terraform (for cloud infrastructure)
cd terraform
terraform init
terraform plan
terraform apply

5. Deploy Carbonio

# Install Carbonio Playbook
ansible-galaxy collection install zxbot.carbonio_install

# Run deployment
cd ansible
ansible-playbook site.yml -u root -i inventory/carbonio-inventory

6. Monitor

Access Prometheus/Grafana:

kubectl port-forward svc/prometheus 9090:9090
kubectl port-forward svc/grafana 3000:3000

Or with Docker:

docker exec -it carbonio-monitoring firefox http://localhost:3000

Configuration

Environment Variables

Create a .env file:

# Carbonio
CARBONIO_VERSION=25.9.0
CARBONIO_DOMAIN=mail.example.com
LDAP_PASSWORD=SecurePassword123!

# Docker
DOCKER_SUBNET=172.20.0.0/16
NODE1_IP=172.20.0.10
NODE2_IP=172.20.0.20
# ... etc

# Kubernetes
K8S_NAMESPACE=carbonio
K8S_REPLICAS=1

# Prometheus
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
RETENTION_DAYS=30

# Resources
CPU_LIMIT=2000m
MEMORY_LIMIT=2Gi
CPU_REQUEST=500m
MEMORY_REQUEST=512Mi

Customization

Each component can be customized:

  • Docker: Modify docker-compose.yml and docker/Dockerfile.ubuntu
  • Kubernetes: Edit YAML files in kubernetes/
  • Terraform: Adjust terraform/terraform.tfvars and modules
  • Ansible: Customize roles in ansible/roles/
  • Monitoring: Update prometheus/prometheus.yml and Grafana dashboards

Deployment Steps

Step 1: Infrastructure Setup

# Initialize infrastructure
./scripts/setup.sh

# Validate infrastructure
./scripts/validate.sh

Step 2: Docker Containers

# Build and start containers
docker-compose -f docker/docker-compose.yml build
docker-compose -f docker/docker-compose.yml up -d

# Verify containers
docker ps
docker logs carbonio-node1

Step 3: Ansible Deployment

# Install dependencies
ansible-galaxy collection install zxbot.carbonio_install
ansible-galaxy install -r ansible/roles-galaxy/requirements.yml

# Deploy core node first
ansible-playbook ansible/deploy-core.yml -u root -i ansible/inventory/carbonio-inventory

# Deploy all nodes
ansible-playbook ansible/site.yml -u root -i ansible/inventory/carbonio-inventory -v

Step 4: Monitoring Setup

# Deploy Prometheus and Grafana
kubectl apply -f kubernetes/monitoring/ # or
docker-compose -f prometheus/docker-compose.yml up -d

# Access dashboards
# Prometheus: http://localhost:9090
# Grafana: http://localhost:3000 (admin/admin)

Step 5: Verification

# Health check
./scripts/health-check.sh

# Check all services
zmprov gs $(hostname) | grep Status

Operations

Scale Nodes

# Scale Docker containers
docker-compose -f docker/docker-compose.yml up -d --scale mailstore=3

# Scale Kubernetes replicas
kubectl scale deployment/carbonio-mailstore --replicas=3 -n carbonio

Update Configuration

# Ansible playbook for updates
ansible-playbook ansible/site.yml -u root -i ansible/inventory/carbonio-inventory --tags=update

# Rolling restart
ansible-playbook ansible/site.yml -u root -i ansible/inventory/carbonio-inventory --tags=restart

Backup and Restore

# Create backups
docker exec carbonio-node1 /opt/scripts/backup.sh

# Restore from backup
docker exec carbonio-node1 /opt/scripts/restore.sh <backup_file>

Health Monitoring

# Monitor all nodes
./scripts/health-check.sh

# Check specific service
docker exec carbonio-node1 zmprov gs $(hostname)

# View logs
docker logs -f carbonio-node1
kubectl logs -f deployment/carbonio-core -n carbonio

Troubleshooting

See TROUBLESHOOTING.md for detailed troubleshooting guides:

  • Container networking issues
  • Kubernetes pod scheduling
  • SSH connectivity problems
  • Carbonio service errors
  • Database connection failures
  • Prometheus scrape failures

Monitoring and Alerts

Prometheus Metrics

  • Node metrics: CPU, Memory, Disk, Network
  • Docker metrics: Container CPU, Memory, I/O
  • Carbonio metrics: Mail throughput, User activity, Queue depth
  • Database metrics: Connections, Queries, Cache hit rate

Grafana Dashboards

  1. Carbonio Overview: System and application health
  2. Node Exporter: Detailed node metrics
  3. PostgreSQL: Database performance
  4. Docker Stats: Container resource usage
  5. Kubernetes: Cluster health (if using K8s)

Alert Rules

Configure alerts for:

  • High CPU/Memory usage
  • Disk space running out
  • Service down/unhealthy
  • Replication lag
  • Mail queue backup
  • Database connection pool exhaustion

Performance Tuning

See ARCHITECTURE.md for:

  • Memory allocation recommendations
  • Storage I/O optimization
  • Network bandwidth planning
  • Database tuning
  • Cache configuration

Security

  • Change default LDAP password
  • Configure firewall rules between nodes
  • Use SSH keys (not passwords)
  • Enable Carbonio SSL certificates
  • Secure PostgreSQL connections
  • Restrict Prometheus access

Support

Contributing

Contributions are welcome! Please:

  1. Fork the repository
  2. Create a feature branch
  3. Make your changes
  4. Test thoroughly
  5. Submit a pull request

License

MIT License - See LICENSE file for details

Changelog

Version 1.0.0 (Current)

  • Initial 7-node Full Standard deployment
  • Kubernetes orchestration support
  • Terraform infrastructure code
  • Docker containerization
  • Ansible automation
  • Prometheus/Grafana monitoring
  • Complete documentation

Author

Created for deploying Carbonio CE 25.9.0 infrastructure

Related Resources

Last Updated: November 2025

About

No description, website, or topics provided.

Resources

Readme

License

MIT license
Activity
Custom properties

Stars

0 stars

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages