Since now several days everything is running fine and all our metrics are green.
Posted Nov 07, 2017 - 22:34 CET
Monitoring
Analysis / Debrief :
During an DNS operation from our service provider an error was done. This DNS propagation was starting to impact SlimPay slowly around 07:10AM CET.
Infrastructure team started their investigation soon alert was received from SlimPay monitoring (from 7:13AM to 8:20AM). When we saw this issue was impacting all our client (based on the morning traffic) we contacted our service provider and got several exchange about it (between 8:30AM CET to 9:23AM CET). Following this discussion an incident was raised on SlimPay statuspage (9:23AM CET). Infrastructure team provided an temporary solution at 10:10AM CET in waiting a final solution from our service provider. At 11:52 our service provider found the root cause and applied an emergency fix.
Root cause: Our service provider did a DNS night operation which caused this service outage.
Next step: Until tomorrow we'll continue to use the temp solution tonight and tomorrow morning we'll see if the fix deployed by our service provider is well propagated before to rollback on it.
Posted Nov 02, 2017 - 17:57 CET
Update
Our infrastructure team has identified the root cause. It look like a DNS issue on our signature provider.
In waiting a resolution from our signature provider a temp fix has been set up by our infrastructure team. Service should be resumed now.
Every details (start time, end time, root cause, resolution) will be communicated in the next update
Posted Nov 02, 2017 - 10:10 CET
Investigating
We have detected some potential trouble on our signature provider. We are investigating on it and will provide update soon.