On the advice of the vendor’s hardware support, Pummelzacken was re-installed. There is some doubt about whether this really has a chance of fixing the problems, but is necessary to eliminate some possibilities and move on.
The failed drive was replaced and the RAID array re-built. The delayed carto style update was pushed 2014-09-06.
Ramoth has suffered a couple of problems recently - one when postgres locked up due to a corrupt lock table and then a few days later when the machine locked up and had to be power cycled.
The main database array on ramoth is nearly full, so we plan to switch katla to be the master server and then reformat ramoth to make more space.
We got 4 new HP DL360 G6s, which are racked but not yet powered at UCL, waiting on sorting out provision of extra power there. The new machines will likely replace older G4ps and G5s which can then be retired.
We had a conference call meeting on 29th September. A summary is available.
The call was held over WebRTC using talky.io, with mixed results. Video had to be disabled to preserve bandwidth, and particular audio connections would fail leaving some people only hearing one half of a conversation, forcing a reconnect. We continue to seek alternatives while awaiting progress on http://iswebrtcreadyyet.com/