Throttling Control - QuotaHourUsed

This space is made available to users of Open Patent Services (OPS) web-service and now also to users of EPO’s bulk data subscription products such as 14. EPO worldwide bibliographic database (DOCDB), 14.11 EPO worldwide legal status database (INPADOC), 14.12 EP full text data, 14.1 EP bibliographic data (EBD)and more.

Users can ask each other questions, exchange experiences and solutions, post ideas. The moderator will use this space to announce changes or other relevant information.
Post Reply

martien
Posts: 31
Joined: Tue Jul 16, 2013 1:45 pm
Contact:

Throttling Control - QuotaHourUsed

Post by martien » Tue May 22, 2018 7:32 pm

Hi all,

in https://forums.epo.org/throttling-control-robots-7587 you explained a lot about throttling.
Meantime I still see the following issues:

1. hourly quota is dependent on the calling server-IP and not the user (OPS Consumer).
On my test servers I got this using the same key/secret:
Server 1:
20180522 19:29:34 0 QuotaWeekUsed: 932553421; QuotaHourUsed: 20783765
About same time, server 2:
20180522 19:29:35 0 QuotaWeekUsed: 932553600; QuotaHourUsed: 129596

The weekly quota is not dependent on the IP and works as expected.

A concept, where every user on a server
- has it's own OPS-Key and secret to serve OPS-queries to serve
- can get data according the fair use policy,
therefore doesn't work.

Question: Is the QuotaHourUsed definitely dependent on the IP-address?
If yes, can that be corrected (in future)?


2. throttling control on searches cannot be controlled completely.
Also with controlled access it may happen, that one node delivers "black" and asks for waiting (retry_after), while another node reports "green" with a positive number of possible requests/min.

Question: So how do I decide after getting a 0 req/min and then a lot of green with 30 req/min?
Is it possible to get the throttling info of all nodes?

In the following data you can see this:
(sometime we get throttling control twice, but with the same info)
"1526785172.5725" "search" "30" "yellow" "1" "2.0"
"1526785175.01376" "search" "30" "green" "0" "2.0"
"1526785175.01376" "search" "30" "green" "1" "2.0"
"1526785175.94604" "search" "15" "red" "0" "4.0"
"1526785175.94604" "search" "15" "red" "1" "4.0"
"1526785182.17471" "search" "30" "green" "0" "2.0"
"1526785182.17471" "search" "30" "green" "1" "2.0"
"1526785183.14857" "search" "0" "black" "0" "900.0"
"1526785243.584" "search" "0" "black" "0" "839.568"
"1526785304.09696" "search" "0" "black" "0" "779.055"
"1526785364.54621" "search" "30" "green" "0" "2.0"
"1526785364.54621" "search" "30" "green" "1" "2.0"
"1526785364.91865" "search" "30" "green" "0" "2.0"
"1526785364.91865" "search" "30" "green" "1" "2.0"
"1526785365.26036" "search" "0" "black" "0" "717.892"
"1526785425.68235" "search" "30" "green" "0" "2.0"
"1526785425.68235" "search" "30" "green" "1" "2.0"
"1526785426.05138" "search" "30" "green" "0" "2.0"
"1526785426.05138" "search" "30" "green" "1" "2.0"
"1526785426.47559" "search" "30" "green" "0" "2.0"
"1526785426.47559" "search" "30" "green" "1" "2.0"
"1526785426.86287" "search" "30" "green" "0" "2.0"
"1526785426.86287" "search" "30" "green" "1" "2.0"
"1526785427.21671" "search" "0" "black" "0" "655.934"
"1526785487.72337" "search" "0" "black" "0" "595.43"
"1526785548.19401" "search" "30" "green" "0" "2.0"


Thanks for your answers in advance!

Regards, Martien


martien
Posts: 31
Joined: Tue Jul 16, 2013 1:45 pm
Contact:

Re: Throttling Control - QuotaHourUsed

Post by martien » Wed May 23, 2018 9:32 am

additional to QuotaHourUsed, also the situation
'overloaded (images=green:50, inpadoc=green:30, other=green:1000, retrieval=green:50, search=black:0)'
seems to be IP dependent.

While I got this on one testserver, the other was still running with the same Key/Secret.
Is this OK?


Post Reply