Notice: My personal stance on AI generated artwork. Retweet and share if you agree. Let us discuss, and not immediately scream bloody murder.

Now Viewing: API is returning favorites sorted by their ID, not added date (again)
Keep it civil, do not flame or bait other users. If you notice anything illegal or inappropriate being discussed, contact an administrator or moderator.

Hawker - Group: Member - Total Posts: 42
user_avatar
API is returning favorites sorted by their ID, not added date (again)
Posted on: 02/12/24 12:12AM

Seen this weird behavior before (gelbooru.com/index.php?page=forum&s=view&id=6614), and it's back again. Quite simply:

curl "gelbooru.com:443/index.ph...mit=10&pid=2329"

{"@attributes":{"limit":10,"offset":23290,"count":23296},"favorite":[{"id":278465889,"user_id":17627,"favorite":9291665,"added":1707675353},{"id":278465890,"user_id":17627,"favorite":9318569,"added":1707675353},{"id":278465891,"user_id":17627,"favorite":9321469,"added":1707675354},{"id":278465893,"user_id":17627,"favorite":9321471,"added":1707675354},{"id":278465896,"user_id":17627,"favorite":9327161,"added":1707675354},{"id":278465972,"user_id":17627,"favorite":9327654,"added":1707675384},{"id":278465973,"user_id":17627,"favorite":9327655,"added":1707675385},{"id":278465974,"user_id":17627,"favorite":9327656,"added":1707675385},{"id":278465975,"user_id":17627,"favorite":9327764,"added":1707675385},{"id":278465976,"user_id":17627,"favorite":9341259,"added":1707675385}]}

Note that these favorite IDs are strictly increasing. Compare this to the last ten ids shown on my "recent favorites" page:
9515291,9521307,9521308,9558893,9565397,9417624,9430335,9454715,9490226,9397010

Which is a clearly different set. FWIW, starting with an offset of zero starts at the "bottom" of my favorites by ID, and not favorited time as shown on my favorites page.

This is an issue, as it prevents gallery-dl (or anything else really) from downloading only the recent additions; if I want to be sure I got everything then I have to check the entire favorites list, which is a few hundred API calls needlessly.



Hawker - Group: Member - Total Posts: 42
user_avatar
Posted on: 02/12/24 08:05AM

...and now the "pid=0" call is returning the "top" of the stack rather than the bottom, but still sorted by post ID, not added time. Of course this is worse from gallery-dl's perspective, as now it's trying to look for "new" items from what is essentially closer to the oldest favorites rather than the newest. But it's still not the added order (in either sort direction), which means having to examine the entire favorites list to get anything, which then means getting blocked (rather appropriately) by the API for abuse.



Hawker - Group: Member - Total Posts: 42
user_avatar
Posted on: 02/13/24 08:40PM

API still doesn't return results in the same order as the Favorites page (i.e. date added descending, what is expected and what the web site itself uses for the favorites page).

I just want to sync my latest favorites to local disk, but I can't do that if the API doesn't sort them properly. I don't want to scan the whole list, you don't want me to scan the whole list - it's dumb.

Gallery-dl (using the favorites API and clearly starting at the "end", which should be the latest additions):
"GET /index.php?page=post&s=view&id=3842359 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3842758 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3843034 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3843434 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3843901 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3844086 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3845636 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3846086 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3846793 HTTP/1.1" 200 None
"GET /index.php?page=post&s=view&id=3846794 HTTP/1.1" 200 None

curl:
curl -s "gelbooru.com:443/index.ph...mit=10&pid=2347" | jq | grep favorite
"favorite": [
"favorite": 3846086,
"favorite": 3845636,
"favorite": 3844086,
"favorite": 3843901,
"favorite": 3843434,
"favorite": 3843034,
"favorite": 3842758,
"favorite": 3842359,



ThePigeon - Group: Member - Total Posts: 5701
user_avatar
Posted on: 02/13/24 08:47PM

This sounds like something you should put on the Trac.



Hawker - Group: Member - Total Posts: 42
user_avatar
Posted on: 02/14/24 08:39AM

TIL. Will do!



add_replyAdd Reply


1