Thread for your script-related questions

[Resu]

When I try to run apgluxe 5.x, it responds with this:

Code: Select all

-bash: ./apgluxe: No such file or directory

Why does it do this?

[PK22]

When I try to run apgluxe 5.x, it responds with this:

Code: Select all

-bash: ./apgluxe: No such file or directory

Why does it do this?

You need to run ./recompile.sh first.

[dvgrn]

When I try to run apgluxe 5.x, it responds with this:

Code: Select all

-bash: ./apgluxe: No such file or directory

Why does it do this?

In cases like this it's always good to start with the theory that the error message is telling you exactly what is wrong.

Try the bash command ls (and hit Enter). Is there a file called "apgluxe" listed in the current bash directory?

If not, then that explains why you're getting that error message.

It doesn't explain why "apgluxe" is missing from the directory you're looking at. Maybe you need to learn how to use the "cd" change-directory command to point to the correct directory -- or maybe you already know how to do that.

Nobody is going to be able to guess what your specific difficulty is unless you give much more detailed information with your question. The best level of detail would be a completely specific walkthrough of everything you did to set up apgsearch, to get to the point where you are now.

[Resu]

[Contents of dvgrn's post removed to save space...]

Seems to be working now, but no idea why...
Edit: Segmentation faults appeared

[PK22]

How exactly do I use transfer.py, besides running python3 transfer.py? Do you have to call it for specific apgcodes, and if so, how? Or can you just run it in general?

[Resu]

When I try to use apgseach, segmentation fault appears:

Code: Select all

$ cd apgmera
./recompile.sh
./apgluxe -n 10000000 -k #anon
sh -c 'rm -f apgluxe *.o */*.o *.op */*.op *.gdca */*.gcda *.profraw *.profdata || true'
echo Clean done
Clean done
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
Ensuring lifelib is up-to-date...
Rule unspecified; assuming b3s23.
Symmetry unspecified; assuming C1.
Configuring rule b3s23; symmetry C1
Using /usr/bin/python3 to configure lifelib...
Valid symmetry: C1
Success!
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  main.cpp -o main.o
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  includes/md5.cpp -o includes/md5.o
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  includes/happyhttp.cpp -o includes/happyhttp.o
g++ -pthread -flto  main.o includes/md5.o includes/happyhttp.o    -o apgluxe
lto-wrapper: warning: using serial compilation of 11 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information
true
true                                                oo o
true                                                oo ooo
true                                                      o
true                                                oo ooo
true                                                 o o
true                                                 o o
true                                                  o
./recompile.sh: line 85:  1574 Segmentation fault      ./apgluxe --rule $rulearg --symmetry $symmarg
Segmentation fault

How do I fix this?

[vilc]

How exactly do I use transfer.py, besides running python3 transfer.py? Do you have to call it for specific apgcodes, and if so, how? Or can you just run it in general?

Running transfer.py is not a fully automated task, many steps of pre/post-processing have to be done manually. Here is what I have figured out so far :

(I assume that you already have Shinjuku installed.)
The easiest way to use transfer.py is probably through paratran.py (that you can find at the root of Shinjuku), which is a wraper script for the function 'synthesise_things'. It works like this :

Code: Select all

python paratran.py <triples> <apgcodes> <out>

Where :

• "triples" is the name of a file containing components in a generic form (called 'triples'). There is a file "shinjuku/triples.txt" that comes with Shinjuku but it wasn't updated in two years. You can generate your own set of triples with these commands :

  Code: Select all

  from shinjuku.search import read_components
  from shinjuku.transfer import components_to_triples
  
  shinjuku_lines = read_components() # add a list of files as a parameter if you don't want to parse the entire database.
  # do some filtering here if you want
  triples = components_to_triples(shinjuku_lines) # converts the lines into a set of triples
  # store triples in a file / merge them in an existing file
  

  This is a slow process, so you may want to generate such a file once and for all, and then merge new triples in your file as you update the database. I am currently in the process of rebuilding my own list, so I can post it soon if someone is interested.
• "apgcodes" is a file containing the apgcodes of the still-lifes that you want to process, one per line.
• "out" is a .sjk file that will receive all possible applications of the components you provided to the set of still-lifes. This file can become very big (170MB for all xs17 for example...), so for large searches you will need to make a more evolved script than paratran.py.

After you generated the new components applications, you have to process them separately in order to filter those which are of interest for you (keep in mind that the majority of the results will be useless because transfer.py is not looking at cost minimisation).

If you want to get the current costs of all objects according to the database, you have to first update the database using 'compile-dedupe.py' to get the latest results from Catagolue. Also check from time to time
https://gitlab.com/parclytaxel/Shinjuku ... mp/autogen for new manually-uploaded files. You can then use the function "dijkstra" from shinjuku.search, which creates a dictionnary that maps apgcodes to (total cost, predecessor, component) tuples.

I may have forgotten some details here. It is better if you also look into some parts of the code by yourself, there is some documentation in the comments. The most useful functions are in transcode.py, transfer.py and search.py.

Also, you can try this parallelised version of transfer.py, made by Alex Greason :
https://github.com/AlexGreason/Transfer.
I never managed to make it work on my end, but it should be much more efficient.

Edit the day after : added latest list of triples.
Edit 2: I must correct what I said about transfer.py not being able to minimise costs. You can pass a "min_paths" dictionnary to synthesise_things (but not in paratran.py). I just never use this since my computer cannot hold both min_paths and the components tree in memory at the same time (both are several gigabytes large if you are working on the entire database).

[Resu]

How do I fix this? (see quoted post)

[wildmyron]

How do I fix this? (see quoted post)

It's not entirely clear what's going on because the input commands are mixed up with the output - but it looks to me that that the command to run apgluxe is being interpreted as arguments to recompile.sh. Try running a single command at a time rather than pasting all of them into the console at once. Also, it is generally always helpful to give more background information: OS; shell environment; how you installed the problematic program/how you obtained the source code; and in the case of compiling a program which compiler and version you are using.

[Resu]

It's not entirely clear what's going on because the input commands are mixed up with the output - but it looks to me that that the command to run apgluxe is being interpreted as arguments to recompile.sh. Try running a single command at a time rather than pasting all of them into the console at once. Also, it is generally always helpful to give more background information: OS; shell environment; how you installed the problematic program/how you obtained the source code; and in the case of compiling a program which compiler and version you are using.

Windows 11, cygwin and git.

[wildmyron]

It's not entirely clear what's going on because the input commands are mixed up with the output - but it looks to me that that the command to run apgluxe is being interpreted as arguments to recompile.sh. Try running a single command at a time rather than pasting all of them into the console at once. Also, it is generally always helpful to give more background information: OS; shell environment; how you installed the problematic program/how you obtained the source code; and in the case of compiling a program which compiler and version you are using.

Windows 11, cygwin and git.

Thank you. Presumably you used the standard git clone to get a copy of apgmera. I suspect there's no issue there, but we might come back to that later.

I don't use cygwin so I can't confirm my hypothesis. Did you try running the commands one at a time? Please share the output here if that still doesn't work properly.

[Resu]

Thank you. Presumably you used the standard git clone to get a copy of apgmera. I suspect there's no issue there, but we might come back to that later.

I don't use cygwin so I can't confirm my hypothesis. Did you try running the commands one at a time? Please share the output here if that still doesn't work properly.

$ cd apgmera

Code: Select all

$  cd apgmera

user@computer ~/apgmera
$

$ ./recompile.sh

Code: Select all

$ ./recompile.sh
sh -c 'rm -f apgluxe *.o */*.o *.op */*.op *.gdca */*.gcda *.profraw *.profdata || true'
echo Clean done
Clean done
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
grep: warning: stray \ before -
Ensuring lifelib is up-to-date...
Rule unspecified; assuming b3s23.
Symmetry unspecified; assuming C1.
Configuring rule b3s23; symmetry C1
Using /usr/bin/python3 to configure lifelib...
Valid symmetry: C1
Success!
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  main.cpp -o main.o
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  includes/md5.cpp -o includes/md5.o
g++ -c -Wall -Wextra -pedantic -O3 -pthread -g -march=native -flto --std=c++11  includes/happyhttp.cpp -o includes/happyhttp.o
g++ -pthread -flto  main.o includes/md5.o includes/happyhttp.o    -o apgluxe
lto-wrapper: warning: using serial compilation of 11 LTRANS jobs
lto-wrapper: note: see the ‘-flto’ option documentation for more information

Later:

Code: Select all

true
true                                                oo o
true                                                oo ooo
true                                                      o
true                                                oo ooo
true                                                 o o
true                                                 o o
true                                                  o
./recompile.sh: line 85:  1140 Segmentation fault      ./apgluxe --rule $rulearg --symmetry $symmarg

[eRroR_6o6]

Why does attempting to change directories to dr give "no such file or directory", when I clearly have a folder named dr?

[PK22]

When searching stdin symmetries with apgsearch v5, how does it ensure that the RLE of the 'soups' can be recovered from Catagolue, and what part(s) of apgsearch v1.1 will I need to change so that it can do the same if I implement stdin symmetries?

[Resu]

Where does apgsearch v1.1 Py3 store the symmetry used?
I can't seem to find it.

[PK22]

Where does apgsearch v1.1 Py3 store the symmetry used?
I can't seem to find it.

I don't remember implementing storage of symmetries, what do you mean?

[Resu]

I don't remember implementing storage of symmetries, what do you mean?

No, I mean storing which symmetry the user chose to search.

[Katrina]

I want to make my own version of gfind/qfind/etc. with floating rows and Generations/HROT support. The reason why I am trying to make one (for tons of CA) myself is because the only other program that supports HROT and generations is really slow, and fails to implement non-vertical speeds properly (even though it's meant to work with diagonal/oblique speeds). Also, the paper that documents gfind's algorithm is really confusing, and I can't understand how the algorithm works (Most of the times, whenever I try to write my own search program, it is insanely slow and buggy, and it is mainly due to this). Can someone help me understand these algorithms? (without the BFS/DFS part because I already mostly know how that works)

[yujh]

apgsearch does not seem to work correctly after failing to upload a haul, is this problem known?

see https://catagolue.hatsya.com/haul/b3-cy ... y6ikn7e/C1 , the number of objects vs soups is inconsistent.

[PK22]

apgsearch does not seem to work correctly after failing to upload a haul, is this problem known?

see https://catagolue.hatsya.com/haul/b3-cy ... y6ikn7e/C1 , the number of objects vs soups is inconsistent.

The problem might be poor internet connection - most of the haul sizes are multiples of 20000, and if apgsearch cannot contact Catagolue, it will continue searching.

[yujh]

apgsearch does not seem to work correctly after failing to upload a haul, is this problem known?

see https://catagolue.hatsya.com/haul/b3-cy ... y6ikn7e/C1 , the number of objects vs soups is inconsistent.

The problem might be poor internet connection - most of the haul sizes are multiples of 20000, and if apgsearch cannot contact Catagolue, it will continue searching.

Take a look at soups/objects for small hauls and large hauls. This value should be around the same, but it does not appear to be.

[PK22]

Is there a faster way to get the population of a pattern over time using lifelib?

Code: Select all

def searchsoup(instring):
    hexadecimal = str(hashlib.sha256(instring.encode('utf-8')).hexdigest())[0:16]
    pt = lt.pattern(rle=hextorle(hexadecimal))
    populationlist = []
    for x in range(100):
        populationlist.append(pt[x].population)

[Katrina]

I want to make my own version of gfind/qfind/etc. with floating rows and Generations/HROT support. The reason why I am trying to make one (for tons of CA) myself is because the only other program that supports HROT and generations is really slow, and fails to implement non-vertical speeds properly (even though it's meant to work with diagonal/oblique speeds). Also, the paper that documents gfind's algorithm is really confusing, and I can't understand how the algorithm works (Most of the times, whenever I try to write my own search program, it is insanely slow and buggy, and it is mainly due to this). Can someone help me understand these algorithms? (without the BFS/DFS part because I already mostly know how that works)

Someone PLEASE help me find what's wrong with this code,
it is driving me insane how slow it is, and I cannot seem to fix it. Also, it can only do period 1 searches for some stupid reason.

[vilc]

Someone PLEASE help me find what's wrong with this code,
it is driving me insane how slow it is, and I cannot seem to fix it. Also, it can only do period 1 searches for some stupid reason.

No one can give you much help if you don't comment your code and clean it up before posting.
For example, your code would be easier to read if all the long NUMSTATE * NUMSTATE * ... products were replaced with named constants or alternatively constexpr functions "square", "cube", etc. applied to NUMSTATE. Try to shorten your lines with intermediate variables or linebreaks (100 characters long might be readable but 295 is way too much). You should also move the code that prints patterns in a separate function.

It is difficult to compare the speed of your program with qfind/gfind/... while the only allowed period is 1. What makes you say that it is slow ?

Anyway, I analysed your program with a profiler, which revealed that it spends most of its time inside the "Path::next" function and its children (around 100ns for each call, but millions of times per second), perhaps something to work on.

[Resu]

How does apgsearch detect when a soup has stabilized?
Is there a search program for finding welds?