Mars Express observations temporarily suspended
31 October 2011
Anomalies in the operation of the solid-state mass memory system on board Mars Express have caused science observations to be temporarily halted. A technical work-around is being investigated that will enable the resumption of a number of observations and should evolve into a long-term solution.Update: 24 November 2011
Mission controllers are making excellent progress in returning Mars Express to routine service. Some science activities have already resumed after being temporarily suspended last month following a series of faults related to the onboard data storage system. For further details see "Mars Express steadily returns to routine operation" in the right-hand menu.
In mid-August, Mars Express autonomously entered safe mode, an operational mode designed to safeguard both the spacecraft itself and its instrument payload in the event of faults or errors.
 |
|
Mars Express has been in orbit around the Red Planet for almost eight years. Credit: ESA / Alex Lutkus |
Switch to spare memory controller
As the previous safe mode was three years ago and the current event looked like a 'normal' transition to safe mode, the flight control team executed the standard recovery procedure and restarted observations. A few days later, another fairly similar set of SSMM problems occurred. The decision was then taken to switch over to the cold-redundant spare, or 'B-side', SSMM controller, as this was virtually the only subsystem common to the two events. The decision to act was triggered by both the need to achieve stable science performance and the need to reduce the consumption of fuel caused by transitions to safe mode.
An important element in trying to reduce the number of safe mode transitions, besides the science observing time being lost, is the amount of fuel used when entering safe mode, which is approximately equal to the amount of fuel required for six months of nominal operations. The reason for this fuel consumption when entering safe mode is the 'Sun acquisition' process, which requires a significant amount of spacecraft manoeuvring.
Full science observations were resumed on 15 September, using the B-side SSMM controller. Eight days later, the spacecraft again entered safe mode. The cause was a checksum error during communication between two subsystems within the SSMM. As this event exhibited no similarities with the two events in August, it was decided to treat this as an isolated incident and return to nominal operations; this took place on 29 September.
On 11 October, a similar problem occurred, but no safe mode transition was triggered. As all operations continued normally, it was decided to take no action, but to find a slot to execute a warm reset of the SSMM at the earliest opportunity.
| Mars Express mass memory anomaly – event timeline | |
| Date | Event |
| 13 August 2011 | Safe mode transition following SSMM read/write error |
| 18 August 2011 | Observations restarted |
| 23 August 2011 | Re-occurrence of SSMM read/write error – without safe mode transition |
| 24 August 2011 | Decision to switch to B-side SSMM controller |
| 15 September 2011 | Observations restarted using B-side SSMM controller |
| 23 September 2011 | Safe mode transition following SSMM internal communications error |
| 29 September 2011 | Return to nominal operations |
| 11 October 2011 | SSMM internal communications error – without safe mode transition |
| 16 October 2011 | Safe mode transition following SSMM internal communications error Observations temporarily suspended |
Observations halted
On Sunday, 16 October, Mars Express again entered safe mode. The cause was a sequence of events very similar to the one that triggered the safe mode transition on 23 September, following an internal communication error.
Resuming use of the A-side SSMM controller is not considered a viable option; it would most probably lead to further safe mode transitions caused by memory module read/write errors and the additional fuel consumption caused by these events cannot be sustained. Faced with these circumstances, the Mission Manager decided to halt science observations.
Possible work-around
The SSMM is a critical subsystem on board Mars Express, central to all platform and instrument operations. The ESOC flight control team is preparing a work-around to address the current problems and allow at least partial resumption of science observations.
During normal operations, telecommands for the platform and instrument payload that have been received from the ground stations are stored in the Long Mission TimeLine (L-MTL), a special file in the SSMM that is used to fill the platform's command cache. In order to cope with anomalous operation scenarios, a hardware-based timeline store – the Short Mission TimeLine (S-MTL) – is available outside the SSMM subsystem.
The work-around now being investigated uses the S-MTL to store telecommands before they are executed.
Good progress is being made with the implementation of this alternative approach to commanding the Mars Express platform and its payload. A full test on the spacecraft will be executed in the near future.
ESA's mission control team and other experts are working closely with the designers and manufacturers of the SSMM subsystem to investigate the cause of the read/write anomalies and to attempt to find a solution that will enable a return to normal operations.
Contact
F. Jansen, Mars Express Mission Manager
Directorate of Science and Robotic Exploration, ESA, The Netherlands
Email: fjansen
rssd.esa.int
