The sound distributed version control system

#391 can't clone - ZSTD("Unknown frame descriptor")

Closed on March 8, 2021
tankf33der on March 7, 2021

Tested on different Linux distroes with different versions of zstd (1.4.5-1.4.9), macos (1.4.9).

None systems survived on scenario clone-loop.sh, sometimes it clones (!!!).

$ ./clone-loop.sh 
+ cd ..
+ rm -rf pjtemp
+ (( i=0  ))
+ (( i<4 ))
+ pijul clone https://nest.pijul.com/pijul/pijul pjtemp
Downloading changes [=================================================] 371/371
           Applying [=================================================] 371/371
 Completing changes [                                                 ] 0/0    
+ rm -rf pjtemp
+ (( i=i+1  ))
+ (( i<4 ))
+ pijul clone https://nest.pijul.com/pijul/pijul pjtemp
Downloading changes [=======>                                         ] 58/371 
           Applying [=======>                                         ] 58/371 
Downloading changes [=======>                                         ] 58/371 
           Applying [=======>                                         ] 58/371 
Error: Changestore error: ZSTD("Unknown frame descriptor")
+ exit 1
$

It almost always fails in different places, debug logs in progress.

tankf33der on March 7, 2021

Example run Nr.1:

https://envs.sh/AS.txt

Also note:

$ cat AS.txt | egrep -v "DEBUG|INFO"
[2021-03-07T17:24:26Z ERROR pijul::remote::http] Error while downloading "https://nest.pijul.com/pijul/pijul/.pijul", retrying
[2021-03-07T17:24:26Z ERROR pijul::remote::http] Error while downloading "https://nest.pijul.com/pijul/pijul/.pijul", retrying
Error: Changestore error: ZSTD("Unknown frame descriptor")

Example run Nr2:

https://envs.sh/AW.txt

pmeunier on March 8, 2021

Adding the following line just after the first pijul rec panics 100% of the time:

/home/…/pijul/target/debug/pijul debug > /dev/null

I have absolutely no idea of why we never saw that before.

tankf33der on March 8, 2021

Project dont have required number of power users and current community provides tiny coverage.

pmeunier on March 8, 2021

Oops, I posted that to the wrong discussion (I meant it for #393). Other than that, I agree with you. I hope that the beta will bring in more people, but the hard thing to do is that it needs to be stable enough before we can call it beta. We’ll get there!

pmeunier added a change on March 8, 2021
EW22B3ZNOWHB33QZPH277NJ4XRDL3XOXSY7IDTVKICVTIX2DFNHQC
pmeunier added a change on March 8, 2021
R4RRU65V3ERJBPKTBBZDOQ4PE3F7UPYMVUBPFPF5VVTFIWTWTJMAC
main
pmeunier on March 8, 2021

Alright, simulating a very slow connection using sudo tc qdisc add dev wlan0 root tbf rate 256kbit latency 200ms burst 1540, I was able to reproduce the problem 6 times out of 10.

The attached patch gets it down to 0 in 10 tests.

pmeunier closed this discussion on March 8, 2021
tankf33der on March 8, 2021

Confirm. Macos ok too.

pmeunier on March 9, 2021

Thanks.