Silk discussion thread

For scripts to aid with computation or simulation in cellular automata.
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Silk discussion thread

Post by Anivec »

Edit by Sokwe (moved to the top of the thread by apgoucher): this is a discussion thread for the drifter search program Silk.
dvgrn wrote: June 11th, 2024, 10:41 pm Okay, in honor of the rather impressive new Spartan G-to-H that showed up in Silk search results yesterday, here's a random completion of a G-to-G2 at repeat time 112, with a 120x115 bounding box:
Is Silk search currently available?
User avatar
dvgrn
Moderator
Posts: 12023
Joined: May 17th, 2009, 11:00 pm
Location: Madison, WI
Contact:

Re: Thread for your script-related questions

Post by dvgrn »

AlbertArmStain wrote: June 12th, 2024, 12:52 pm Is Silk search currently available?
There's a Silk git repository, currently under active development with two contributors (apgoucher plus some clever catalyst hacking by Mitchell Riley). The version in the master branch should be in pretty good working order, I would think.

We should probably have a LifeWiki tutorial that walks through the basic setup, at least as far as being able to, say, re-find the Snark in a couple of minutes.
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Thread for your script-related questions

Post by Anivec »

dvgrn wrote: June 12th, 2024, 2:30 pm There's a Silk git repository, currently under active development with two contributors (apgoucher plus some clever catalyst hacking by Mitchell Riley). The version in the master branch should be in pretty good working order, I would think.

We should probably have a LifeWiki tutorial that walks through the basic setup, at least as far as being able to, say, re-find the Snark in a couple of minutes.
One can imagine that it is a GPU accelerated search intended for supercomputers/high-performance computing by the image of a British prime minister holding four graphics processing units and what seems to be 14 supercomputer cabinets in the background for the image. With the use of CUDA in programing makes the search less ideal for normal computers, meaning that I'm not sure I should download this on my computer because my specs might fall under the minimum.

Will a 3.3 GHz 6-Core Intel Core i5, AMD Radeon Pro 5300 4 GB and 32 GB of RAM at 2667 MHz cut it? (I am not exactly sure how many graphics cards I have but I know I have less than 5).

Also, is the download process the same as Barrister's?
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Thread for your script-related questions

Post by Anivec »

Can someone at least tell me how to get Silk Search up and running?
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Thread for your script-related questions

Post by apg »

AlbertArmStain wrote: June 13th, 2024, 8:53 pm Can someone at least tell me how to get Silk Search up and running?
The easiest way is to run it in the cloud by using this Google Colab notebook, as that doesn't require you to have an NVIDIA GPU:

https://colab.research.google.com/drive ... sp=sharing

Colab will generally give you a couple of hours of GPU time each day (in their free tier). 5 minutes of that will be spent compiling Silk, but the rest you can usefully use to run your searches.
dvgrn wrote: June 12th, 2024, 2:30 pm We should probably have a LifeWiki tutorial that walks through the basic setup, at least as far as being able to, say, re-find the Snark in a couple of minutes.
2 minutes was with an RTX 4090 (high-end consumer GPU). The GPUs available in the free tier of Google Colab are Tesla T4 GPUs, which are about 6x slower -- so it should take 10-15 minutes on one of those.

If, on the other hand, you do have your own NVIDIA GPU, then it's much more convenient to run your searches locally. If you're running Windows, then you'll want to use WSL2 to give you a Linux environment (at least that's what whatislife used to run Silk to discover the LWSS-to-G). Basically you want to run the same commands that the Colab notebook does:

Code: Select all

git clone https://gitlab.com/apgoucher/silk.git
silk/recompile.sh
silk/build/src/silk --help
What do you do with ill crystallographers? Take them to the mono-clinic!
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Thread for your script-related questions

Post by Anivec »

calcyman wrote: June 15th, 2024, 9:01 pm
AlbertArmStain wrote: June 13th, 2024, 8:53 pm Can someone at least tell me how to get Silk Search up and running?
The easiest way is to run it in the cloud by using this Google Colab notebook, as that doesn't require you to have an NVIDIA GPU:

https://colab.research.google.com/drive ... sp=sharing

Colab will generally give you a couple of hours of GPU time each day (in their free tier). 5 minutes of that will be spent compiling Silk, but the rest you can usefully use to run your searches.
Thank you for your help, I still have a few questions:
Is there any other tools needed that are not packaged into Silk? Also, can you put the Google collab in the READ ME for Silk? Finally, is CUDA compatible with iOS? I’ve been getting mixed results for compatibility with iOS. Thanks for helping again.
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Thread for your script-related questions

Post by apg »

AlbertArmStain wrote: June 16th, 2024, 10:28 am Thank you for your help, I still have a few questions:
Is there any other tools needed that are not packaged into Silk?
It has dependencies on CMake and nvcc -- but if you're using the Google Colab then you won't need to worry about installing anything, because it doesn't run on your computer; it runs on one of Google's cloud VMs where these things are preinstalled.
Finally, is CUDA compatible with iOS? I’ve been getting mixed results for compatibility with iOS.
If you're using Google Colab, then you should be able to run Silk from any browser (for the reason mentioned above); you don't need CUDA installed on your computer because the code is running elsewhere.

If you're trying to run Silk on your own machine, then apparently CUDA doesn't work on MacOS (and newer Macs don't have NVIDIA GPUs in any case): https://developer.nvidia.com/nvidia-cud ... -mac-hosts
What do you do with ill crystallographers? Take them to the mono-clinic!
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Thread for your script-related questions

Post by Anivec »

calcyman wrote: June 16th, 2024, 11:28 am
AlbertArmStain wrote: June 16th, 2024, 10:28 am Thank you for your help, I still have a few questions:
Is there any other tools needed that are not packaged into Silk?
It has dependencies on CMake and nvcc -- but if you're using the Google Colab then you won't need to worry about installing anything, because it doesn't run on your computer; it runs on one of Google's cloud VMs where these things are preinstalled.
Finally, is CUDA compatible with iOS? I’ve been getting mixed results for compatibility with iOS.
If you're using Google Colab, then you should be able to run Silk from any browser (for the reason mentioned above); you don't need CUDA installed on your computer because the code is running elsewhere.

If you're trying to run Silk on your own machine, then apparently CUDA doesn't work on MacOS (and newer Macs don't have NVIDIA GPUs in any case): https://developer.nvidia.com/nvidia-cud ... -mac-hosts
Thanks for your support!
I have a few more, probably less important questions about Silk:
1. Why did you choose the name “Silk” over the remaining Hunting of the Snark characters
2. Why did you decide to attach an image of someone with a powdered wig into the image files, who is depicted there?

And the more important questions:
1. Would you like me to give some improvement suggestions? I see Silk is a work in progress. I’d love to help you out with some ideas for features
2. How did you find the G-to-Wing-to-Herschel? It seems like a conduit that would be found with Catforce, not Barrister.
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Thread for your script-related questions

Post by apg »

AlbertArmStain wrote: June 18th, 2024, 9:47 am 1. Why did you choose the name “Silk” over the remaining Hunting of the Snark characters
There are two complementary reasons:
  • The term 'silk' refers to a senior barrister (at least in England and Wales), so this is apt for an improved* version of Barrister: https://castleassociates.org.uk/ask-the ... gal-system
  • There are existing weaving-related metaphors in CUDA, such as warps and threads, so the name 'Silk' is especially fitting for a CUDA port of Barrister..
*of course it's only 'improved' in the sense of much greater throughput (about 1000x faster on a 4090 than running Barrister on the CPU); Barrister has many more features. I expect that Silk will continue to gradually adopt features from Barrister if and when they're needed for some experiment.
2. Why did you decide to attach an image of someone with a powdered wig into the image files, who is depicted there?
I asked GPT-4 to generate an image of a senior barrister inside a datacentre, holding a GPU in each hand, with an aura of green light -- and I was pleasantly surprised how well it was able to do that! (The robes, bands, and wig are indeed part of a traditional barrister's dress.)
1. Would you like me to give some improvement suggestions? I see Silk is a work in progress. I’d love to help you out with some ideas for features
Potentially in the future, yes. The best way to contribute to Silk is to fork the repository on GitLab, make the changes yourself, and then create a pull request to update the main Silk repository. (This is what mvr did for the catalyst hack, for example.)

There are currently three things that are on the immediate roadmap for me:
  1. Adding more documentation and explanation about the inner workings of Silk: there are plenty of ideas that went into Silk, some largely inspired by Barrister and others (such as the vectorised priority heap) being relevant to parallelising it to run efficiently on GPUs. I think that this could be shaped into a series of articles that teach advanced GPU programming in the context of building Silk.
  2. Multistream support: this is an upcoming change to the search strategy. Instead of just doing parallel DFS throughout, this will have a preprocessing stage that performs a more BFS-like search to partition the search space so that it can be distributed amongst multiple GPUs. It will even improve single-GPU performance for large searches on powerful hardware (such as a RTX 4090) by ensuring that the GPU is never starved of work.
  3. Bloom filter: this is an experimental optional feature that will attempt to deduplicate 'essentially equivalent' areas of search space, such as a Herschel conduit search where an eater1 or eater2 could be used in a certain location. It will be at the expense of completeness (it could, in rare cases, miss some solutions, e.g. because it 'selects' an eater1 and then later the active region reinteracts in a way that would have required an eater2), so disabled by default, but I think that it would be able to maximise novelty found per unit compute expended.
Until those are done (most likely over the course of a month or two, because most of my time is taken up with my main job so I can only sporadically contribute to Silk), I'm very unlikely to implement any new features myself (unless they're utterly trivial to implement) so contributing features yourself through a GitLab pull request is really the only way to get new features into Silk any time soon.
2. How did you find the G-to-Wing-to-Herschel? It seems like a conduit that would be found with Catforce, not Barrister.
Silk found the following eater when I started a search (not finished; I aborted it because it was running into a combinatorial explosion, but will revisit when the Bloom filter feature is enabled) for glider eaters with active bounds 14 14 14 and the Snark explicitly excluded from the search space:

Code: Select all

x = 29, y = 30, rule = LifeHistory
5.A14.A$4.2BA11.3A$4.3AB9.A$5.4B8.2A$.2A3.4B4.5B3.2A$2.A4.4B2.4B5.A$
2.A.AB.5B.5B.BA.A$3.2AB.11B.B2A$5.15B$5.15B$2A5.13B$.A5.13B$.A.AB.16B
5.2A$2.2AB.17B4.A$4.10B2A7B.BA.A$3.11B2A7B.B2A$3.22B$4.21B$3.2D20B$4.
DBD16B$5.2D2.10B2.4B$11.5B.B5.2A$11.6B6.A$12.7B5.3A$5.2A5.4B2A.A6.A$
5.2A7.B2.A.3A$13.A.A.A4.A$13.2A.A.4A$16.A2.A$16.2A!
I noticed that the wing output (which I misidentified as a century before wwei corrected me) could attach to a known W-to-H. Shortly afterwards, iNoMed and I independently discovered that the complicated catalyst can be replaced with an eater1 (and Silk subsequently discovered the same thing later in the search), at which point it became a (dependent) Thessalonic G-to-H.
What do you do with ill crystallographers? Take them to the mono-clinic!
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Thread for your script-related questions

Post by Anivec »

calcyman wrote: June 19th, 2024, 9:02 am
Until those are done (most likely over the course of a month or two, because most of my time is taken up with my main job so I can only sporadically contribute to Silk), I'm very unlikely to implement any new features myself (unless they're utterly trivial to implement) so contributing features yourself through a GitLab pull request is really the only way to get new features into Silk any time soon.
I was going to give a suggestion about how it could have this feature that act similar to a filter making it easier to find general elementary conduits and signal connections, especially 2c/3 wire connections. I guess that's for another time though.
I'll post it here so I don't forget

Take this pattern as an input and specify the offset you want. Let the red cells be where the active cells of the catalyst might be present, and the blue cells represent where the white colored catalyst can be placed in along with its signal needed to be activated by some catalyst at any time. The blue cells can also contain some of the active catalyst cells.

Code: Select all

x = 54, y = 51, rule = LifeHistory
10.23D$10.23D$10.23D$10.23D$10.23D$10.23D$10.23D$33D$33D$33D$33D$33D$
33D$33D$33D$33D21B$33D21B$33D21B$33D21B$18D36B$18D2B6C28B$18DBC6BC27B
$12.7BC2B5C27B$12.5BCBCBC7BC24B$12.5B2C2BC2B6C24B$12.9BCBC30B$12.8B2C
BC2B6C22B$12.11BCBC6BC21B$12.11BCBCBED4C21B$12.12B2CBC7BC18B$12.15BC2B
6C18B$12.15BCBC24B$12.14B2CBC2B6C16B$12.17BCBC6BC15B$12.17BCBC2B5C15B
$12.18B2CBC7BC12B$12.21BC2B6C12B$12.21BCBC18B$12.20B2CBC2B6C10B$12.23B
CBC6BC9B$12.23BCBC2B5C9B$12.24B2CBC14B$12.27BC14B$12.27BC14B$12.26B2C
14B$12.42B$12.42B$12.42B$12.42B$12.42B$12.42B!
Now you specify the input, which, in this case is going to be a glider

Code: Select all

x = 3, y = 3, rule = LifeHistory
.A$2.A$3A!
And the glider has a placement range on the specified origin of:

Code: Select all

x y w h
Which is equivalent to the "search-area" feature in Catforce.
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Thread for your script-related questions

Post by apg »

AlbertArmStain wrote: June 19th, 2024, 1:09 pm and the blue cells represent where the white colored catalyst can be placed
The problem is that Silk is much more like dr than like CatForce. MathAndCode explained it very eloquently here:
MathAndCode wrote: October 7th, 2020, 9:06 pm There are two ways to think about ConwayLife: cell-based and pattern-based. The cell-based approach focuses on the individual cells and whether they're on or off. The individual cells do not move. They merely turn on or off. The pattern-based approach focuses on entire patterns. Unlike the cell-based approach, the pattern-based approach allows objects, such as gliders, to move.
Silk is entirely cell-based, and for a good reason: it means that all of the logic can be bitsliced and vectorised. Suggestions that involve placing catalysts aren't really compatible with how Silk works internally.
What do you do with ill crystallographers? Take them to the mono-clinic!
simeks
Posts: 429
Joined: March 11th, 2015, 12:03 pm
Location: Sweden

Silk discussion thread

Post by simeks »

Edit by Sokwe: this is a discussion thread for the drifter search program Silk.

With this amazing new tool we will need a Silk thread now right, so I went ahead and created one.
Previous Silk-related posts started here

I can report that I was able to compile and run Silk on an Amazon AWS g6 instance with no issues.

I used this AMI:
Deep Learning Base OSS Nvidia Driver GPU AMI (Ubuntu 22.04) 20240610

If you use Ctrl-C to stop a search you might need to do:
> sudo fuser -v /dev/nvidia*

And then do:
kill <PID>

on the PID that says Silk, or else GPU memory won't be released.
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

simeks wrote: June 20th, 2024, 2:09 pm With this amazing new tool we will need a Silk thread now right, so I went ahead and created one.
Previous Silk-related posts started here
Thanks for creating this thread! I've taken the liberty of moving the posts from the other thread here.

There's also a (more active) discussion on Discord: https://discord.com/channels/3579222555 ... 6579435692
What do you do with ill crystallographers? Take them to the mono-clinic!
User avatar
Anivec
Posts: 1984
Joined: January 28th, 2022, 7:18 pm
Location: In 4.3 miles, take a right onto Exit 54

Re: Silk discussion thread

Post by Anivec »

A couple more questions:
  1. What is the maximum size of the torus the pattern is on?
  2. Can Silk be ran on multiple threads?
Also, thanks for your support
simeks
Posts: 429
Joined: March 11th, 2015, 12:03 pm
Location: Sweden

Re: Silk discussion thread

Post by simeks »

Given this input file:

Code: Select all

x = 48, y = 48, rule = LifeHistory
13$14.A$15.A$13.3A5$20.4B$20.4B$20.4B$20.4B7$30.2A$30.2A!
Silk run with:
> build/src/silk -s 6 in.rle 4 4 2

finds:

Code: Select all

# Found catalysis
..............o.................................................
...............o................................................
.............ooo................................................
................................................................
................................................................
................................................................
................................................................
....................**..........................................
....................*.*.........................................
......................*.........................................
......................**........................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
..............................oo................................
..............................oo................................
as expected, but increasing -s to 7, or removing the -s option, for example:
> build/src/silk -s 7 in.rle 4 4 2

that result isn't found anymore.
I don't think that can be the expected behaviour.
Can anyone else replicate this?
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

simeks wrote: June 24th, 2024, 6:25 am or removing the -s option, for example:
> build/src/silk -s 7 in.rle 4 4 2

that result isn't found anymore.
I don't think that can be the expected behaviour.
Can anyone else replicate this?
Did you mean to draw the block in white (state 3) so that it's treated as a catalyst? Otherwise the block is considered to be an input, same as the glider, so (without the -s option) the expected behaviour is to not return that result as it does not fully consume the input.

As for the sensitivity to the -s parameter, it does indeed behave quite strangely (and needs to be fixed) because of how cycle detection interacts with it.
AlbertArmStain wrote: June 22nd, 2024, 12:07 pm What is the maximum size of the torus the pattern is on?
The torus size is 64x64 (it's not just a maximum; it's hardcoded very deeply into Silk).
Can Silk be ran on multiple threads?
I'm not sure that I understand the question? Silk runs on a GPU, and GPUs are highly parallel processors, and the speed advantage of Silk versus Barrister is primarily a result of the increased parallelism. For example, an RTX 4090 GPU (which was capable of discovering the Snark in < 2 minutes) has 128 streaming multiprocessors, each with 4 partitions -- so it's like having 512 "cores" each with "AVX-1024" (not an official term, but rather a description of the SIMD width), and Silk uses all of this parallelism. That's why, on an RTX 4090, Silk is about 1000x faster than Barrister; it's just that the hardware has so much more (parallel) computing power than a single CPU core.

I'm currently working on a branch to enable Silk to use more than one GPU if you have multiple GPUs available; that would unlock a further level of parallelism beyond what is currently supported.
What do you do with ill crystallographers? Take them to the mono-clinic!
simeks
Posts: 429
Joined: March 11th, 2015, 12:03 pm
Location: Sweden

Re: Silk discussion thread

Post by simeks »

calcyman wrote: June 24th, 2024, 6:52 am Did you mean to draw the block in white (state 3) so that it's treated as a catalyst?
No, sorry I should have mentioned that, I put it there as part of the input pattern just to have a known result when no catalysts are placed in the blue area, because of the grid being a torus.
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

simeks wrote: June 24th, 2024, 7:28 am
calcyman wrote: June 24th, 2024, 6:52 am Did you mean to draw the block in white (state 3) so that it's treated as a catalyst?
No, sorry I should have mentioned that, I put it there as part of the input pattern just to have a known result when no catalysts are placed in the blue area, because of the grid being a torus.
I think that block is what's causing the problem, and in any case it's not necessary (Silk uses Floyd's cycle detection algorithm, so it will recognise when it becomes periodic with p256).
What do you do with ill crystallographers? Take them to the mono-clinic!
simeks
Posts: 429
Joined: March 11th, 2015, 12:03 pm
Location: Sweden

Re: Silk discussion thread

Post by simeks »

calcyman wrote: June 24th, 2024, 9:46 am I think that block is what's causing the problem, and in any case it's not necessary (Silk uses Floyd's cycle detection algorithm, so it will recognise when it becomes periodic with p256).
I tried that now, and removing the block does help for that particular problem.

Could it be that this problem occurs when the active pattern is converted to a stable pattern that is non-empty, except for the catalysts?
I tried with this input file instead, designed to find a snake boat-bits catalyst:

Code: Select all

x = 48, y = 48, rule = LifeHistory
13$14.A$15.A$13.3A9$20.4B$20.4B$20.4B$20.4B!
and here I still find:

Code: Select all

# Found catalysis
..............o.................................................
...............o................................................
.............ooo................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
.....................**.........................................
......................*.........................................
.....................*..........................................
.....................**.........................................
when using "-s 6" or lower, but starting at 7, no result.

Here is the output in case that is useful:

Code: Select all

build/src/silk -s 6 in.rle 4 4 2
+---------+-----------------------------------+---------+---------+-------------------+-----------------------------+
| elapsed |           problems                | current | rollout | speed (Mprob/sec) |          solutions          |
|  clock  +---------------+-------------------+  batch  |   per   +---------+---------+---------+---------+---------+
|   time  |     solved    |  open (pct full)  |   size  | problem | current | overall | recover | oscill. | fizzles |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
|  0.10 s |            21 |        0 ( 0.00%) |       4 |   6.591 |   0.000 |   0.000 |       2 |       0 |       0 |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+

# Found catalysis
..............o.................................................
...............o................................................
.............ooo................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
.....................**.........................................
......................*.........................................
.....................*..........................................
.....................**.........................................


         1                                               0                   1                      
original problems                             *--> restabilised   *--> low-period                   
        |                                    /                    |                                 
        |                           1       /            1        |          0                0     
        |                *---> periodic ---*-----> oscillatory ---*--> high-period --*--> completed 
        v               /                   \                                        |   oscillators
        21             /            8        \           0                   0       |       
problems processed ---*---> contradictory     *--> fizzled out ---*--> completed     |       
        ^             |                                           |     fizzles      |       
        |             |             2                       1     |                  |       
        20            *---> catalyst recovered ---*--> completed  |                  |       
  new problems        |                           |    catalyses  |                  |       
      ^   ^           |            10             |               |                  |       
      |   |           *---> indeterminate         |               v          1       v       
      |   |                        |              *--> determined uncompletable by SAT solver
      |   |             10         |                                    
      |   *-------- bifurcated <---*                                    
      |                 |          |                    0               
      *-----------------*          *---> deduplicated by a Bloom filter 

build/src/silk -s 7 in.rle 4 4 2
+---------+-----------------------------------+---------+---------+-------------------+-----------------------------+
| elapsed |           problems                | current | rollout | speed (Mprob/sec) |          solutions          |
|  clock  +---------------+-------------------+  batch  |   per   +---------+---------+---------+---------+---------+
|   time  |     solved    |  open (pct full)  |   size  | problem | current | overall | recover | oscill. | fizzles |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
|  0.10 s |            21 |        0 ( 0.00%) |       4 |   6.591 |   0.000 |   0.000 |       0 |       0 |       0 |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+


         1                                               2                   1                      
original problems                             *--> restabilised   *--> low-period                   
        |                                    /                    |                                 
        |                           3       /            1        |          0                0     
        |                *---> periodic ---*-----> oscillatory ---*--> high-period --*--> completed 
        v               /                   \                                        |   oscillators
        21             /            8        \           0                   0       |       
problems processed ---*---> contradictory     *--> fizzled out ---*--> completed     |       
        ^             |                                           |     fizzles      |       
        |             |             0                       0     |                  |       
        20            *---> catalyst recovered ---*--> completed  |                  |       
  new problems        |                           |    catalyses  |                  |       
      ^   ^           |            10             |               |                  |       
      |   |           *---> indeterminate         |               v          0       v       
      |   |                        |              *--> determined uncompletable by SAT solver
      |   |             10         |                                    
      |   *-------- bifurcated <---*                                    
      |                 |          |                    0               
      *-----------------*          *---> deduplicated by a Bloom filter 

User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

simeks wrote: June 24th, 2024, 10:24 am
calcyman wrote: June 24th, 2024, 9:46 am I think that block is what's causing the problem, and in any case it's not necessary (Silk uses Floyd's cycle detection algorithm, so it will recognise when it becomes periodic with p256).
I tried that now, and removing the block does help for that particular problem.

Could it be that this problem occurs when the active pattern is converted to a stable pattern that is non-empty, except for the catalysts?
Yes, that's because by default it discards oscillators and still-lifes. If you set -p 1 then it will report everything, including restabilisations:

Code: Select all

build/src/silk examples/ekstrom.rle 4 4 2 -p 1
Info: Silk invoked as build/src/silk
Info: loaded 1 subproblems.
Info: initial perturbation has 5 cells.
Info: probing 1 devices...
    -- device 0 has 3599 MiB free and 4023 MiB total.
Info: creating 2 streams...
    -- creating stream 0 on device 0 with ring buffer size 524288
    -- creating stream 1 on device 0 with ring buffer size 524288
Info: all comms established; commencing search...
+---------+-----------------------------------+---------+---------+-------------------+-----------------------------+
| elapsed |           problems                | current | rollout | speed (Mprob/sec) |          solutions          |
|  clock  +---------------+-------------------+  batch  |   per   +---------+---------+---------+---------+---------+
|   time  |     solved    |  open (pct full)  |   size  | problem | current | overall | recover | oscill. | fizzles |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
|  0.05 s |            21 |        0 ( 0.00%) |       4 |   6.591 |   0.000 |   0.000 |       0 |       3 |       0 |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+

# Found oscillator with period 256
..............o.................................................
...............o................................................
.............ooo................................................

# Found restabilisation
..............o.................................................
...............o................................................
.............ooo................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
.....................**.........................................
......................*.........................................
.....................*..........................................
.....................**.........................................


         1                                               2                  -2                      
original problems                             *--> restabilised   *--> low-period                   
        |                                    /                    |                                 
        |                           3       /            1        |          3                2     
        |                *---> periodic ---*-----> oscillatory ---*--> high-period --*--> completed 
        v               /                   \                                        |   oscillators
        21             /            8        \           0                   0       |       
problems processed ---*---> contradictory     *--> fizzled out ---*--> completed     |       
        ^             |                                           |     fizzles      |       
        |             |             0                       0     |                  |       
        20            *---> catalyst recovered ---*--> completed  |                  |       
  new problems        |                           |    catalyses  |                  |       
      ^   ^           |            10             |               |                  |       
      |   |           *---> indeterminate         |               v          1       v       
      |   |                        |              *--> determined uncompletable by SAT solver
      |   |             10         |                                    
      |   *-------- bifurcated <---*                                    
      |                 |          |                    0               
      *-----------------*          *---> deduplicated by a Bloom filter
What do you do with ill crystallographers? Take them to the mono-clinic!
Sokwe
Moderator
Posts: 3376
Joined: July 9th, 2009, 2:44 pm

Re: Silk discussion thread

Post by Sokwe »

calcyman wrote: June 24th, 2024, 10:53 am by default it discards oscillators
What is the reason for this? I would think it would be nice to output the higher-period (p>10) oscillators by default, as long as checks could be done to discard trivial oscillators and oscillators with known rotors (like with dr's knownrotors file). I worry that a few nice oscillators in the range p10 to p20 could end up being discarded in a fizzle search.
-Matthias Merzenich
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

Sokwe wrote: June 25th, 2024, 7:35 pm
calcyman wrote: June 24th, 2024, 10:53 am by default it discards oscillators
What is the reason for this? I would think it would be nice to output the higher-period (p>10) oscillators by default, as long as checks could be done to discard trivial oscillators and oscillators with known rotors (like with dr's knownrotors file). I worry that a few nice oscillators in the range p10 to p20 could end up being discarded in a fizzle search.
You can provide -p 11 (for example) to output all oscillators with period >= 11. I find that it usually spams the console with lots of trivial oscillators (e.g. p42 oscillators formed from a non-interacting burloaferimeter and $rats) so there would need to be a postprocessing step -- probably before the existing stator completion step -- to remove those. I'll need to think how to quickly determine whether an oscillator is trivial though.
What do you do with ill crystallographers? Take them to the mono-clinic!
Sokwe
Moderator
Posts: 3376
Joined: July 9th, 2009, 2:44 pm

Re: Silk discussion thread

Post by Sokwe »

calcyman wrote: June 25th, 2024, 8:27 pm I'll need to think how to quickly determine whether an oscillator is trivial.
How quick does it need to be? Are there so many that it might take a substantial portion of the CPU time?

dr constructs the rotor descriptor, determines whether the rotor is made up of separate connected rotors, and if so, reports the distance between them. It doesn't check for triviality, but I suppose each connected rotor could be tested on its own to see if it has a proper factor period. I don't know how fast this would be, or how easy it would be to implement. I don't have a machine that can run Silk at the moment, so I'm not really in a position to be requesting features anyway.
-Matthias Merzenich
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

Sokwe wrote: June 26th, 2024, 3:28 pm
calcyman wrote: June 25th, 2024, 8:27 pm I'll need to think how to quickly determine whether an oscillator is trivial.
How quick does it need to be? Are there so many that it might take a substantial portion of the CPU time?
Absolutely, yes! For example this found 8272 oscillators of period >= 11:

Code: Select all

+---------+-----------------------------------+---------+---------+-------------------+-----------------------------+
| elapsed |           problems                | current | rollout | speed (Mprob/sec) |          solutions          |
|  clock  +---------------+-------------------+  batch  |   per   +---------+---------+---------+---------+---------+
|   time  |     solved    |  open (pct full)  |   size  | problem | current | overall | recover | oscill. | fizzles |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
| 50.87 s |      25638388 |        0 ( 0.00%) |       8 |   7.856 |   0.513 |   0.504 |       0 |    8272 |      50 |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
On an RTX 4090 GPU, that amount of search space is covered in 1 second, not 50, so even with 8 CPU cores processing solutions you'd need to be able to do the processing in < 1 millisecond per oscillator per core just to keep up. Stator completion/minimisation takes a couple of seconds so this processing would need to happen before we complete the stator.
What do you do with ill crystallographers? Take them to the mono-clinic!
User avatar
apg
Moderator
Posts: 3007
Joined: June 1st, 2009, 4:32 pm

Re: Silk discussion thread

Post by apg »

There have been a few updates to Silk today:
  • mvr has added new options --exempt-existing (a boolean flag that excludes catalysts in the input file from counting towards the active region bounds, matching Barrister's default behaviour) and --max-perturbed-time N (an integer to limit how long the catalyst can interact before stabilising).
  • Silk can now parallelise the search across multiple GPUs, not just one (and will automatically use all available GPUs in the system). In practice you will see the speed quickly ramp up in the first few seconds of the search as the GPUs coordinate to share work:

Code: Select all

+---------+-----------------------------------+---------+---------+-------------------+-----------------------------+
| elapsed |           problems                | current | rollout | speed (Mprob/sec) |          solutions          |
|  clock  +---------------+-------------------+  batch  |   per   +---------+---------+---------+---------+---------+
|   time  |     solved    |  open (pct full)  |   size  | problem | current | overall | recover | oscill. | fizzles |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
|    1 s  |      12543273 |   796436 (18.99%) |  258048 |   7.788 |  12.347 |  12.347 |       0 |       0 |       3 |
|    2 s  |      32154921 |   659852 (15.73%) |  258048 |   7.760 |  18.987 |  15.694 |       0 |       0 |       3 |
|    3 s  |      58169177 |  2397548 ( 9.53%) | 1363220 |   7.889 |  26.833 |  19.272 |       0 |       0 |       3 |
|    4 s  |     128982667 |  2098550 ( 8.34%) | 1356278 |   7.883 |  71.664 |  32.194 |       0 |       0 |       3 |
|    5 s  |     201032723 |  2189744 ( 8.70%) | 1548288 |   7.889 |  72.448 |  40.199 |       0 |       0 |       4 |
|    6 s  |     276184061 |  1998504 ( 7.94%) | 1548288 |   7.901 |  72.827 |  45.780 |       0 |       0 |       4 |
|    7 s  |     346957049 |  1569076 ( 6.23%) | 1324152 |   7.904 |  73.176 |  49.565 |       0 |       0 |       4 |
|    8 s  |     422387885 |  1704228 ( 6.77%) | 1079624 |   7.928 |  75.384 |  52.794 |       0 |       0 |       4 |
|    9 s  |     494370749 |  1977280 ( 7.86%) | 1548288 |   7.938 |  71.154 |  54.855 |       0 |       0 |       4 |
|   10 s  |     566732979 |  1683390 ( 6.69%) | 1351000 |   7.953 |  72.230 |  56.593 |       0 |       0 |       5 |
+---------+---------------+-------------------+---------+---------+---------+---------+---------+---------+---------+
What do you do with ill crystallographers? Take them to the mono-clinic!
Post Reply