I think I may be running out of RAM and swap and somehow the system locks up and fails to free up any memory. I removed "-pipe" now from my flags and added another 32GB swap file. Let's see.