Page 1 of 1

Throttling Control - QuotaHourUsed

Posted: Tue May 22, 2018 7:32 pm
by martien
Hi all,

in https://forums.epo.org/throttling-control-robots-7587 you explained a lot about throttling.
Meantime I still see the following issues:

1. hourly quota is dependent on the calling server-IP and not the user (OPS Consumer).
On my test servers I got this using the same key/secret:
Server 1:
20180522 19:29:34 0 QuotaWeekUsed: 932553421; QuotaHourUsed: 20783765
About same time, server 2:
20180522 19:29:35 0 QuotaWeekUsed: 932553600; QuotaHourUsed: 129596

The weekly quota is not dependent on the IP and works as expected.

A concept, where every user on a server
- has it's own OPS-Key and secret to serve OPS-queries to serve
- can get data according the fair use policy,
therefore doesn't work.

Question: Is the QuotaHourUsed definitely dependent on the IP-address?
If yes, can that be corrected (in future)?


2. throttling control on searches cannot be controlled completely.
Also with controlled access it may happen, that one node delivers "black" and asks for waiting (retry_after), while another node reports "green" with a positive number of possible requests/min.

Question: So how do I decide after getting a 0 req/min and then a lot of green with 30 req/min?
Is it possible to get the throttling info of all nodes?

In the following data you can see this:
(sometime we get throttling control twice, but with the same info)
"1526785172.5725" "search" "30" "yellow" "1" "2.0"
"1526785175.01376" "search" "30" "green" "0" "2.0"
"1526785175.01376" "search" "30" "green" "1" "2.0"
"1526785175.94604" "search" "15" "red" "0" "4.0"
"1526785175.94604" "search" "15" "red" "1" "4.0"
"1526785182.17471" "search" "30" "green" "0" "2.0"
"1526785182.17471" "search" "30" "green" "1" "2.0"
"1526785183.14857" "search" "0" "black" "0" "900.0"
"1526785243.584" "search" "0" "black" "0" "839.568"
"1526785304.09696" "search" "0" "black" "0" "779.055"
"1526785364.54621" "search" "30" "green" "0" "2.0"
"1526785364.54621" "search" "30" "green" "1" "2.0"
"1526785364.91865" "search" "30" "green" "0" "2.0"
"1526785364.91865" "search" "30" "green" "1" "2.0"
"1526785365.26036" "search" "0" "black" "0" "717.892"
"1526785425.68235" "search" "30" "green" "0" "2.0"
"1526785425.68235" "search" "30" "green" "1" "2.0"
"1526785426.05138" "search" "30" "green" "0" "2.0"
"1526785426.05138" "search" "30" "green" "1" "2.0"
"1526785426.47559" "search" "30" "green" "0" "2.0"
"1526785426.47559" "search" "30" "green" "1" "2.0"
"1526785426.86287" "search" "30" "green" "0" "2.0"
"1526785426.86287" "search" "30" "green" "1" "2.0"
"1526785427.21671" "search" "0" "black" "0" "655.934"
"1526785487.72337" "search" "0" "black" "0" "595.43"
"1526785548.19401" "search" "30" "green" "0" "2.0"


Thanks for your answers in advance!

Regards, Martien

Re: Throttling Control - QuotaHourUsed

Posted: Wed May 23, 2018 9:32 am
by martien
additional to QuotaHourUsed, also the situation
'overloaded (images=green:50, inpadoc=green:30, other=green:1000, retrieval=green:50, search=black:0)'
seems to be IP dependent.

While I got this on one testserver, the other was still running with the same Key/Secret.
Is this OK?