ConwayLife.com

Catagolue is down again, and I don't know why this time.

cordership3 wrote:Catagolue is down again, and I don't know why this time.

It looks like b3678s25678/C1 is straining the server; there are over 500000 distinct objects in there and Anonymous has been uploading hauls every 2 minutes to that census.

EDIT: and is still doing so. Anonymous, please stop.

EDIT 2: I've modified Catagolue to ban this rule for the time being.

Catagolue appears to be down again, but now it's giving a 503 error instead of a 500 error like it was before.

Why didn't B3/S23/1x256X2+1 bring down the site in a few hours, despite me submitting hauls like twice as fast? ?? ??? ???? ????? ?????? ??????? ???????? ????????? ??????????

wwei23 wrote:Why didn't B3/S23/1x256X2+1 bring down the site in a few hours, despite me submitting hauls like twice as fast? ?? ??? ???? ????? ?????? ??????? ???????? ????????? ??????????

As has been explained before, it's really about the number of operations Catagolue needs to perform to process a haul.

Suppose that a rulesym has a very rich repertoire of objects, and is very fast to search, and a user is submitting a haul with, say, 10,000 distinct objects every 2 minutes. That's (at least) 5,000 datastore operations Catagolue has to perform per minute, and since these cost money, the site's daily quota (2 USD, IIRC) will be used up relatively fast.

Suppose that a different rulesym is much less rich in objects, and a user submits hauls of, say, 100 distinct objects every minute. That's 100 datastore operations Catagolue has to perform per minute. I'm sure you can see the difference.

This isn't to say submitting hauls every minute is a good idea, even for less productive rules. IIRC, Calcyman's asked users to not submit hauls more often than once every 20 minutes from a single machine; so if you're running e.g. four searcher instances on that machine, you should adjust haul sizes that you're only submitting hauls once every 80 minutes at most from each.

Depending on the rule this'll reduce server load significantly. Suppose that for the same (second) rulesym considered, the number of distinct objects doubles if you increase the haul size by a factor of ten. Suppose now that you decide to submit a haul every 100 minutes instead of every minute. Hauls will now contain ~400 distinct objects on average, and Catagolue will have to perform ~4 datastore operations per minute and haul.

This is my best understanding of the situation anyway. Calcyman'd be able to tell you more.

(Oh, and you may want to reconsider how many punctuation marks you use.)

How can I search B3678/S25678 without bringing down Catagolue? I don’t know how to create wrapper scripts.
Maybe Catagolue should allow hauls larger than 1 megabyte.

wwei23 wrote:How can I search B3678/S25678 without bringing down Catagolue? I don’t know how to create wrapper scripts.
Maybe Catagolue should allow hauls larger than 1 megabyte.

Wait, you were the one submitting all those "anonymous" B3678/S25678 hauls?

Hauls larger than 1 megabyte would just require more datastore operations, though, so it wouldn't really help except in maybe a few specific rules (and it would make it easier for malicious users to crash Catagolue with rules like B/S012345678).

77topaz wrote:

wwei23 wrote:How can I search B3678/S25678 without bringing down Catagolue? I don’t know how to create wrapper scripts.
Maybe Catagolue should allow hauls larger than 1 megabyte.

Wait, you were the one submitting all those "anonymous" B3678/S25678 hauls?

Hauls larger than 1 megabyte would just require more datastore operations, though, so it wouldn't really help except in maybe a few specific rules (and it would make it easier for malicious users to crash Catagolue with rules like B/S012345678).

Objective frequency drops off exponentially, so larger hauls tend to have less unique objects per megabyte. As a result, it would bee better for Catagolue. Also, when I tried to search B3678/S25678, whenever I put the number of soups high enough to follow 1 haul every 20 minutes, the hauls are too big.

Search movero8 instead then

wwei23 wrote:Also, when I tried to search B3678/S25678, whenever I put the number of soups high enough to follow 1 haul every 20 minutes, the hauls are too big.

And that should be telling you something.

Now, accepting that Catagolue's limits are in place for a reason, I think the salient question isn't "why can't you search B3678/S25678" as much as "why do you want to use Catagolue". Or, perhaps more to the point, "why do you want to use Catagolue to store a FULL census" --- every object, including the entirely of the long tail?

Calcyman's said before that while data fed to Catagolue *can* come from apgsearch, it doesn't have to. As a corollary, it doesn't have to come from an unmodified apgsearch; you could easily rig up a version that only submits part of the result, i.e., the objects considered interesting, whatever that means in practice. (You could also have it save full logs locally, so you'd not lose any data yourself.) So long as you use a rulestring/symmetry that communicates the fact the resulting census isn't a regular, full soup search census, this should be fine, right?

Or alternatively, modify apgsearch so it only saves logs locally and doesn't submit to Catagolue, and use your own tools to hunt for interesting objects.

The "UnbanMajestas" symetry

catagolue is down again. i have my own guesses as to who caused it but i'll leave that in the air.

Hmm, I really think it would be useful to have some sort of feature on the site that showed how much of the daily quota has been used/remained. That would likely help avoid these kinds of outages in the future.

Catagolue is back! I ended up submitting a 78 million-soup haul in b3s23/C1, which is exactly the same as what I got the previous time Catagolue went down.

Oh, and also, looking at the hauls that processed immediately after Catagolue went back up, I think Heavpoot searching b2-a4z678s234w with hauls of ~100 KB and ~2600 distinct objects once per minute definitely contributed to Catagolue going down, and he's still searching that rule as of the time of writing; I recommend raising the threshold for that rule (he was actually warned on the Discord to increase his haul sizes multiple times before this outage).

77topaz wrote:searching b2-a4z678s234w with hauls of ~100 KB and ~2600 distinct objects once per minute

...yeah, that defies common sense, not to mention the ample warning given on the wiki.

Slow down cowboy!

EDIT: Calcyman tells me he's fixed this, by increasing the minimum required for that rule.

Catagolus Descendus!

also wait did they just ad the emoticon i never noticed it

danny wrote:also wait did they just ad the emoticon i never noticed it

Pretty sure I've seen it on Discord for a while now, but not as an animated emoticon because that's restricted to Nitro users.

catagolue is over quota quota-o-meter when

cordership3 wrote: catagolue is over quota quota-o-meter when

Actually, I just refreshed the page, it seems to be back up already!

it's down again

cordership3 wrote:it's down again

Ouch -- someone has been submitting hauls for g6b2ce3ar4a5ns013ij, which has lots of distinct objects. Storing the sample soups was causing a peak of 10000 datastore writes per minute. That incurred a total cost of $0.72 per hour, causing it to soon max out at the daily spending limit of $5.00.

I've now made a commit to the repository to ensure sample soups are not stored for objects in tabulations exceeding 500kB (except for b3s23, which is exempt). That should limit the amount of damage someone can do by intensive searching of a single census.

calcyman wrote:I've now made a commit to the repository to ensure sample soups are not stored for objects in tabulations exceeding 500kB (except for b3s23, which is exempt). That should limit the amount of damage someone can do by intensive searching of a single census.

Does this mean that all existing hauls which take up more than this amount (and, by extension, all rules whose only sample soups lie within such a haul) will be deleted?

muzik wrote:

calcyman wrote:I've now made a commit to the repository to ensure sample soups are not stored for objects in tabulations exceeding 500kB (except for b3s23, which is exempt). That should limit the amount of damage someone can do by intensive searching of a single census.

Does this mean that all existing hauls which take up more than this amount (and, by extension, all rules whose only sample soups lie within such a haul) will be deleted?

No, it merely means that when (for instance) b3678s34678/C1 reached 500 kB of objects in the xp2 tabulation, it will stop including sample soups for that census on individual xp2_??? object pages. The tabulations themselves (including frequencies of each object) are unaffected by this.

The justification is that if a tabulation reaches that size, the individual objects aren't interesting enough to bother recording their sample soups.

What if I'm searching for a certain object, and the tabulation is over 500KB? How will I post the soup in the soup search results in rules other than Conway's Life thread?

wwei23 wrote:What if I'm searching for a certain object, and the tabulation is over 500KB? How will I post the soup in the soup search results in rules other than Conway's Life thread?

Look at the locally-saved log, in particular the @SAMPLE_SOUPIDS section.