Family Services Outages

This space is made available to users of Open Patent Services (OPS) web-service and now also to users of EPO’s bulk data subscription products such as 14. EPO worldwide bibliographic database (DOCDB), 14.11 EPO worldwide legal status database (INPADOC), 14.12 EP full text data, 14.1 EP bibliographic data (EBD)and more.

Users can ask each other questions, exchange experiences and solutions, post ideas. The moderator will use this space to announce changes or other relevant information.
Post Reply

jalipert
Posts: 22
Joined: Wed Nov 15, 2017 5:16 am

Family Services Outages

Post by jalipert » Mon Jun 15, 2020 4:03 am

Hello,

I'm writing about an issue with the OPS Family Services endpoint that we've been experiencing and monitoring for some time.

We frequently experience downtimes of this service several times per week. They seem to be most common on Fridays. These downtimes can last from a few minutes to 30 minutes or more at a time and are having a significant effect on our operations, particularly the longer outages.

I don't know if this is a known issue or if anything can be done about it, but at the least I wanted to bring it to your attention.

As an example, below is a summary of outages that we detected over the past few months (our detection resolution is about 15 minutes, so many outages have likely gone undetected):

[Week]: [Number of Detected outages] (Length of each outage in minutes)

Week of June 7, 2020: 4 outages (1, 2, 15, 30, 75)

Week of May 31, 2020: 5 outages (1, 1, 1, 1, 5)
Week of May 24, 2020: 14 outages (1, 13, 8, 4, 6, 3, 4, 10, 1, 5, 1, 6, 2, 17)
Week of May 17, 2020: 2 outages (1, 4)
Week of May 10, 2020: 2 outages (2, 2)
Week of May 3, 2020: 3 outages (5, 2, 2)

Week of April 26, 2020: 11 outages (2, 6, 10, 27, 1, 1, 35, 3, 8, 3)
Week of April 19, 2020: 3 outages (72, 2, 8)
Week of April 12, 2020: 5 outages (2, 1, 3, 5, 8)
Week of April 5, 2020: 2 outages (7, 3)

Week of March 29, 2020: 22 outages (34, 20, 76, 12, 1, 6, 11, 21, 3, 62, 3, 32, 5, 7, 20, 7, 5, 8, 6, 1, 3, 2)
Week of March 22, 2020: 13 outages (1, 1, 3, 5, 12, 24, 7, 1, 2, 9, 1, 1, 1)
Week of March 15, 2020: 16 outages (1, 1, 105, 36, 7, 4, 7, 5, 49, 72, 12, 1, 138, 1, 1, 4)
Week of March 8, 2020: 1 outage (9)
Week of March 1, 2020: 13 outages (4, 3, 7, 2, 23, 3, 9, 23, 1, 1, 2, 1, 5)

February 2020: 25 outages (11 more than 5 minutes in duration)

January 2020: 35 outages (8 more than 5 minutes in duration)

December 2019: 17 outages (7 more than 5 minutes in duration)

November 15-30, 2019: 14 outages (5 more than 5 minutes in duration)


EPO / OPS Support
Posts: 1074
Joined: Thu Feb 22, 2007 5:32 pm

Re: Family Services Outages

Post by EPO / OPS Support » Mon Jun 15, 2020 6:54 am

Hi,

Are you sure those are real outages? What kind of errors are you getting? Also, let me know who you deal with throttling. Please send me all in a email to patentdata(at)epo.org with your UID so I can check the logs for the times listed below.

Regards,
Vesna for OPS


jalipert
Posts: 22
Joined: Wed Nov 15, 2017 5:16 am

Re: Family Services Outages

Post by jalipert » Mon Jun 15, 2020 4:48 pm

After we started noticing frequent outages, we created a downtime detector that makes a single call to the family API every 15 minutes and checks the result to see if it matches what we expect. Sometimes, the outage takes place by an obvious HTTP/1.1 504 Gateway Timeout error to the family API, and other times, it returns an empty result (I think). I'll add some more logging to our tester so I can try to find a better pattern in what the errors look like.

I do have a log of the last detected Gateway Timeouts (504 errors), which seem to only be every few days:
Sun, 14 Jun 2020 17:09:12 GMT
Fri, 12 Jun 2020 20:59:12 GMT
Mon, 08 Jun 2020 21:03:37 GMT

I'll send you our UID as well via email to see if there's anything you can find out about our account.

Thanks!
Joe


Post Reply