Common Issues

This guide covers common issues when using Noid and their solutions.

Installation Issues

KVM Not Available

Symptoms:

Error: /dev/kvm not found
Error: KVM virtualization not available

Solutions:

  1. Check CPU virtualization support:
egrep -c '(vmx|svm)' /proc/cpuinfo

If output is 0, your CPU doesn't support virtualization or it's disabled in BIOS.

  1. Enable virtualization in BIOS:
  • Reboot and enter BIOS/UEFI
  • Find "Intel VT-x", "AMD-V", or "Virtualization Technology"
  • Enable and save
  1. Load KVM modules:
sudo modprobe kvm
sudo modprobe kvm_intel  # For Intel
# OR
sudo modprobe kvm_amd    # For AMD
  1. Verify KVM:
ls -l /dev/kvm
# Should exist with appropriate permissions

Binary Not Found

Symptoms:

bash: noid: command not found

Solutions:

# Add ~/.local/bin to PATH
echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

# Verify
which noid

Permission Denied

Symptoms:

bash: ./noid: Permission denied

Solutions:

# Make executable
chmod +x ~/.local/bin/noid

# For server
chmod +x ~/.local/bin/noid-server

Server Issues

Server Won't Start

Symptoms:

Error: Failed to start noid-server
Error: Address already in use

Solutions:

  1. Check if already running:
ps aux | grep noid-server
  1. Check port availability (default: 7654):
sudo lsof -i :7654

# Kill if needed
sudo kill -9 <PID>
  1. Check server logs:
sudo journalctl -u noid-server -f
  1. Verify Firecracker installation:
which firecracker
firecracker --version

Connection Refused

Symptoms:

Error: Connection refused to http://server:7654

Solutions:

  1. Verify server is running:
ps aux | grep noid-server
curl http://localhost:7654/health
  1. Check correct port (7654, not 8080):
noid auth setup --url http://your-server:7654 --token <token>
  1. Check firewall:
# Ubuntu/Debian
sudo ufw allow 7654/tcp

# CentOS/RHEL
sudo firewall-cmd --add-port=7654/tcp --permanent
sudo firewall-cmd --reload

Kernel Staleness (CRNG Not Initialized)

Symptoms:

  • TLS connections hang during VM boot
  • VM freezes during initialization
  • Timeouts creating new VMs

Cause: CRNG (cryptographic random number generator) not initialized in guest kernel, especially affects TLS connections.

Solutions:

  1. Use recommended kernel version (6.12.71+):
# Rebuild server with correct kernel
cd noid-cli
sudo bash scripts/install-server.sh
  1. Rebuild golden snapshot:
sudo bash scripts/provision-golden.sh

VM Creation Issues

VM Creation Takes 30-60 Seconds

Cause: No golden snapshot configured - VMs boot from scratch.

Solution:

Create golden snapshot to reduce creation time to 5-10 seconds:

sudo bash scripts/provision-golden.sh

VM Creation Fails

Solutions:

  1. Check server logs:
sudo journalctl -u noid-server -n 100
  1. Verify resources:
free -h
df -h ~/.noid
  1. Check kernel and rootfs paths:
ls -lh /path/to/vmlinux
ls -lh /path/to/rootfs.ext4

Runtime Issues

Command Execution Fails

Symptoms:

Error: Command execution failed
Error: Timeout

Solutions:

  1. Verify VM is running:
noid list
  1. Test simple command:
noid exec my-vm -- echo "test"
  1. Check console for errors:
noid console my-vm
# Press Ctrl+] to exit

Networking Issues

Symptoms:

  • VM has no IP address
  • Cannot reach internet from VM

Solutions:

  1. Verify noid-netd is running:
sudo systemctl status noid-netd
  1. Check IP forwarding:
sysctl net.ipv4.ip_forward
# Should output: net.ipv4.ip_forward = 1

# Enable if needed
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
  1. Check iptables NAT rules:
sudo iptables -t nat -L
  1. Restart network daemon:
sudo systemctl restart noid-netd

IP Address Changes After Restore

Issue: After restoring from checkpoint, VM gets a new IP address.

Solution:

This is expected behavior. Use DNS or service discovery instead of hardcoded IPs:

# Bad: Hardcoded IP
curl http://172.16.0.5:8080

# Good: DNS or hostname
curl http://my-service:8080

Checkpoint/Restore Issues

Golden Snapshot Invalidation

Symptoms:

  • VMs fail to boot from golden snapshot
  • VM creation falls back to slow boot

Cause: Kernel or Firecracker version mismatch.

Solution:

Rebuild golden snapshot after upgrading:

sudo bash scripts/provision-golden.sh

Checkpoint Creation Fails

Solutions:

  1. Check disk space:
df -h ~/.noid
  1. Check VM status:
noid list

Configuration Mismatch

Symptoms: VM creation doesn't use golden snapshot despite it existing.

Cause: VM specs don't match golden snapshot config (default: 1 vCPU, 2048 MiB RAM).

Solutions:

  1. Use default specs:
noid create my-vm  # Uses golden snapshot
  1. Or create new golden with custom specs:

Edit golden snapshot config to match your preferred defaults.

Authentication Issues

Authentication Failed

Solutions:

# Verify token in config
cat ~/.noid/config.toml

# Re-setup with correct credentials
noid auth setup --url http://your-server:7654 --token <token>

Invalid Token

Solutions:

# Request new token from server admin
# Then reconfigure
noid auth setup --url http://your-server:7654 --token <new-token>

Database Issues

Database Locked

Symptoms:

Error: database is locked

Cause: Multiple server instances running.

Solution:

# Check for multiple instances
ps aux | grep noid-server

# Kill extra instances
sudo kill <PID>

Performance Issues

Slow Checkpoint/Restore

Solutions:

  1. Use btrfs for zero-copy:
# Format storage as btrfs
sudo mkfs.btrfs /dev/sdX
sudo mount /dev/sdX /var/lib/noid
  1. Use SSD storage:

Move ~/.noid/ to SSD storage for better performance.

Diagnostic Commands

Essential commands for troubleshooting:

# System check
egrep -c '(vmx|svm)' /proc/cpuinfo
ls -l /dev/kvm
free -h
df -h

# Noid status
noid --version
cat ~/.noid/config.toml
noid list

# Server status
sudo systemctl status noid-server
sudo systemctl status noid-netd
sudo journalctl -u noid-server -n 100

# Network
sysctl net.ipv4.ip_forward
sudo iptables -t nat -L

# Firecracker
which firecracker
firecracker --version

# Golden snapshot
ls -lh ~/.noid/golden/
cat ~/.noid/golden/config.json

Validation Script

Run this comprehensive check:

#!/bin/bash
echo "=== Noid Validation ==="

# KVM
echo -n "KVM: "
[ -e /dev/kvm ] && echo "✓" || echo "✗"

# Firecracker
echo -n "Firecracker: "
which firecracker >/dev/null 2>&1 && echo "✓" || echo "✗"

# Server running
echo -n "Server: "
pgrep noid-server >/dev/null && echo "✓" || echo "✗"

# Network daemon
echo -n "Network daemon: "
pgrep noid-netd >/dev/null && echo "✓" || echo "✗"

# IP forwarding
echo -n "IP forwarding: "
[ "$(sysctl -n net.ipv4.ip_forward)" = "1" ] && echo "✓" || echo "✗"

# Golden snapshot
echo -n "Golden snapshot: "
[ -d ~/.noid/golden ] && [ -f ~/.noid/golden/config.json ] && echo "✓" || echo "✗"

# Config file
echo -n "Client config: "
[ -f ~/.noid/config.toml ] && echo "✓" || echo "✗"

echo "=== End ==="

Getting Help

If these solutions don't work:

  1. Check server logs:

    sudo journalctl -u noid-server -f
    
  2. Report an issue:

  3. Community:

    • GitHub Discussions
    • Project documentation

Next Steps