This document describes the system requirements and recommended configurations for running Scintirete.
Scintirete supports the following operating systems:
Operating System | Architecture | Minimum Version | Recommended Version |
---|---|---|---|
Linux | x86_64 | Ubuntu 18.04, CentOS 7 | Ubuntu 20.04+, CentOS 8+ |
Linux | ARM64 | Ubuntu 18.04, CentOS 7 | Ubuntu 20.04+, CentOS 8+ |
macOS | x86_64 | macOS 10.15 (Catalina) | macOS 12.0+ (Monterey) |
macOS | ARM64 (M1/M2) | macOS 11.0 (Big Sur) | macOS 12.0+ (Monterey) |
Windows | x86_64 | Windows 10 | Windows 11 |
Scintirete stores all vector data in memory to ensure optimal performance. Memory requirements are primarily determined by the following factors:
Total Memory Required = Vector Data + HNSW Index + Metadata + System Overhead
Vector Data Memory = Number of Vectors × Vector Dimension × 4 bytes (float32)
Example Calculations:
HNSW index typically requires an additional 30-50% memory overhead:
Index Memory ≈ Vector Data Memory × 0.3 to 0.5
Vector Count | Dimension | Vector Data | Index Overhead | Recommended Total Memory |
---|---|---|---|---|
10,000 | 768 | ~29 MB | ~12 MB | 512 MB |
100,000 | 768 | ~290 MB | ~120 MB | 1 GB |
1,000,000 | 768 | ~2.9 GB | ~1.2 GB | 8 GB |
10,000,000 | 768 | ~29 GB | ~12 GB | 64 GB |
1,000,000 | 1536 | ~5.7 GB | ~2.3 GB | 16 GB |
/metrics
endpoint to monitor memory usageScintirete is a statically compiled single binary with no additional runtime dependencies required:
Feature | Dependency | Description |
---|---|---|
Docker Deployment | Docker 20.10+ | Containerized deployment |
Monitoring Integration | Prometheus, Grafana | Observability stack |
Text Vectorization | OpenAI API or compatible service | Automatic text embedding |
Reverse Proxy | Nginx, Traefik, etc. | Production load balancing |
Port | Protocol | Purpose | Default | Configurable |
---|---|---|---|---|
HTTP API | TCP | RESTful API service | 8080 | ✅ |
gRPC | TCP | gRPC service | 9090 | ✅ |
Metrics Monitoring | TCP | Prometheus metrics | 8080 (built-in) | ❌ |
Minimal configuration (local access only):
# Allow local loopback access sudo ufw allow from 127.0.0.1
Internal network service configuration:
# Allow internal network access to HTTP API sudo ufw allow from 192.168.0.0/16 to any port 8080 # Allow internal network access to gRPC sudo ufw allow from 192.168.0.0/16 to any port 9090
Public network service configuration (not recommended, use reverse proxy and TLS):
# Caution: Only open public access with appropriate security measures sudo ufw allow 8080 sudo ufw allow 9090
If using text vectorization features, access to OpenAI API is required. Please configure base_url
and api_key
accordingly.
Total Storage Required = Data Files + Log Files + Configuration Files + System Overhead
Component | Storage Required | Description |
---|---|---|
RDB Snapshots | ~Memory data size | Compressed data snapshots |
AOF Logs | Variable | Operation logs, configurable rotation |
Metadata | < 1% of data size | Collection and database metadata |
System Logs | Configurable | Structured logs and audit logs |
Data Scale | Recommended Storage | Storage Type | IOPS Requirement |
---|---|---|---|
< 1GB data | 10 GB | SSD | Basic |
1-10GB data | 100 GB | SSD | Medium |
10-100GB data | 1 TB | NVMe SSD | High |
> 100GB data | Data size × 3 | Enterprise SSD | Very High |
Operating System | Recommended File System | Alternatives |
---|---|---|
Linux | ext4, XFS | Btrfs, ZFS |
macOS | APFS | HFS+ |
Windows | NTFS | ReFS |
# Create dedicated user (recommended) sudo useradd -r -s /bin/false scintirete # Set data directory permissions sudo mkdir -p /var/lib/scintirete sudo chown scintirete:scintirete /var/lib/scintirete sudo chmod 750 /var/lib/scintirete # Set configuration file permissions sudo chmod 640 /etc/scintirete/scintirete.toml sudo chown root:scintirete /etc/scintirete/scintirete.toml
# Increase file descriptor limits echo "* soft nofile 65536" >> /etc/security/limits.conf echo "* hard nofile 65536" >> /etc/security/limits.conf # Optimize memory management echo 'vm.swappiness = 1' >> /etc/sysctl.conf echo 'vm.dirty_ratio = 15' >> /etc/sysctl.conf echo 'vm.dirty_background_ratio = 5' >> /etc/sysctl.conf # Apply configuration sysctl -p
# Limit container memory (prevent OOM) docker run -m 4g scintirete/scintirete:latest # Use local SSD volume docker run -v /fast-ssd/scintirete:/app/data scintirete/scintirete:latest
# Check basic system information uname -a cat /etc/os-release # Check hardware resources free -h # Memory df -h # Disk space lscpu # CPU information
# Test port availability netstat -tuln | grep 8080 netstat -tuln | grep 9090 # Test network latency ping your-scintirete-server
Deployment Mode | Use Case | Resource Requirements | Complexity |
---|---|---|---|
Single Binary | Development, testing, small-scale production | Minimal | Simple |
Docker Container | Standardized deployment | Low | Medium |
Docker Compose | Complete stack with monitoring | Medium | Medium |
Kubernetes | Large-scale, high availability | High | Complex |
Choose the appropriate deployment mode to meet your requirements and resource constraints.