Fixing AWS Lightsail crashing after a few hours

A persistent and annoying issue affects Amazon Web Services (AWS) Lightsail instances: they shut down seemingly randomly after only a few hours. The instance seems fine for an hour or two, then suddenly shuts itself down. This problem is very visible on the web, for example: on AWS rePost, Reddit, and StackOverflow, though none offer a full solution. The Lightsail metrics offered give no clues as to what causes this - CPU usage and network traffic both look normal and well within limits. The issue is due to the memory utilisation, which Lightsail does not surface.

After the instance has crashed once more, reboot it, and take the following steps to diagnose the issue.

Check the logs of the instance by running the command sudo less /var/log/syslog, scrolling back to the time of the crash. Searching for anything that points to out of memory errors - look for "oom". In my case, the culprit was a PHP process /usr/lib/php/sessionclean, which was running on an hourly cron job and apparently had no limits to the amount of memory it could consume.

There are two solutions, and you should do both: tame the PHP sessionclean service; and increase the swap space available for memory usage.

Limiting the PHP sessionclean service can be done by running sudo systemctl edit phpsessionclean.service then limiting the maximum memory of the service to 100 Megabytes by adding:

[Service] MemoryMax=100M

Then restarting the service with.

sudo systemctl daemon-reload sudo systemctl restart phpsessionclean.service

Increasing the swap space can be done by creating a new swap file. Run the following commands to allocate 2GB to a swap file:

sudo fallocate -l 2G /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile echo '/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Verify that the above commands worked by running free -h.

With these two changes in place, your Lightsail instance should stop crashing every few hours and hopefully run reliably, even on the smallest plans.