Three feeds (prologic, movq and mine) and my database is already 1.3 MiB in size. Hmm. I actually got the read filter working. More on that later after polishing it.

@aelaraji@aelaraji.com @mckinley@twtxt.net rsync -avzr with an optional --progress is what I always use. Ah, I could use the shorter -P, thanks @movq@www.uninformativ.de.

We’re now having a thunderstorm with rain, lightning and thunder and the severe weather map shows all green. I’d expect it to be violet.

I’m experimenting with SQLite and trees. It’s going good so far with only my own 439 messages long main feed from a few days ago in the cache. Fetching these 632 rows took 20ms:

Image

Now comes the real tricky part, how do I exclude completely read threads?

It just occurs to me we’re now building some kind of control structures or commands with (edit:…) and (delete:…) into feeds. It’s not just a simple “add this to your cache” or “replace the cache with this set of messages” anymore. Hmm. We might need to think about the consequences of that, can this be exploited somehow, etc.

@movq@www.uninformativ.de Thanks for the summary!

So, what would happen if there is no original message anymore in the feed and you encounter an “edit” subject? Since you cannot verify that the feed contained it in the first place, would you obey it?

Some feed could just make a client update something from a different feed. In the cache, the client would need to store in a flag that this message was updated, so that when it later encounters the message from the real feed, it has a chance of reverting that bogus edit. Hmm. The devil is in the detail.

It’s much easier with a delete subject. When it finds the message in its cache and the feeds match, remove it. Otherwise, just ignore it.

Another thing: At the moment, anyone could claim that some feed contained a certain message which was then removed again by just creating the hash over the fake message in said feed and invented timestamp themselves. Nobody can ever verify that this was never the case in the first place and completely made up. So, our twt hashes have to be taken with a grain of salt.

@prologic@twtxt.net I get where you’re coming from. But is it really that bad in practice? If you follow any link somewhere in the web, you also don’t know if its contents has been changed in the meantime. Is that a problem? Almost never in my experience.

Granted, it’s a nice property when one can tell that it was not messed with since the author referenced it.

Now WTF!? Suddenly, @falsifian@www.falsifian.org’s feed renders broken in my tt Python implementation. Exactly what I had with my Go rewrite. I haven’t touched the Python stuff in ages, though. Also, tt and tt2 do not share any data at all.

By any chance, did you remove the ; charset=utf-8 from your Content-Type: text/plain header, falsifian?

Image

@movq@www.uninformativ.de Non-ASCII characters were broken. Like U+2028, degrees (°), etc.

Turns out I used a silly library to detect the encoding and transform to UTF-8 if needed. When there is no Content-Type header, like for local files, it looks at the first 1024 bytes. Since it only saw ASCII in that region, the damn thing assumed the data to be in Windows-1252 (which for web pages kinda makes sense):

// TODO: change default depending on user's locale?
return charmap.Windows1252, "windows-1252", false

https://cs.opensource.google/go/x/net/+/master:html/charset/charset.go;l=102

This default is hardcoded and cannot be changed.

Trying to be smart and adding automatic support for other encodings turned out to be a bad move on my end. At least I can reduce my dependency list again. :-)

I now just reject everything that explicitly specifies something different than text/plain and an optional charset other than utf-8 (ignoring casing). Otherwise I assume it’s in UTF-8 (just like the twtxt file format specification mandates) and hope for the best.

Hmmmm, I somehow run into an encoding problem where my inserted data end up mangled in the database. But, both SQLite and Go use UTF-8. What’s happening here? :-?

20° temperature drop in just a hand full of days. Ooof. We went on a stroll at 10°C today. I could have used a beanie, my ears were very cold. The sun was out, but hardly any people. Very nice. Also, no wind.

It was nice to finally hear a few birds singing again, although it was still fairly silent. The sun gave us a nice show. In hindsight, we should have stayed at the summit a bit longer. In the forest, we missed the very best, crazy red sky. We could only see parts shimmering through the tree lines.

Image

https://lyse.isobeef.org/waldspaziergang-2024-09-12/

I’m finally continuing with my tt rewrite in Go. So, I thought I use the shiny io/fs.FS. That’s supposed to be a super cool new file system API. It allowed me to write tests more elegantly. I don’t have to place actual test files on disk, but can keep everything nicely in RAM with testing/fstest.MapFS. That actually worked out great, I do like that.

However, os.DirFS("/") for production code is just a terrible solution. I noted that OS paths and io/fs.FS paths are fundamentally different. This new API does not allow leading slashes in the passed paths. This results in an error. So, I have to cut the leading slash off myself.

Also, the whole thing is totally useless on Windows, because of the drives. Simply does not work at all. Well, honestly, I don’t care the slightest bit about that operating system, but it would be nice if this concept were cross-platform.

I haven’t tested it, but I’m pretty sure relative paths or ~ do also not work. I have to first build absolute paths myself. Unfortunately, there is no builtin helper to translate an OS path into a io/fs.FS path.

Of course, others noted these shortcomings and surprising results, too: https://github.com/golang/go/issues/44279 There is no OSFileSystem implementation that would simply allow the easy transition from all the classical os.* functionality to io/fs.FS. And they also do not wanna add something like that either. Sigh.

I’m really wondering what they were thinking when introducing this. :-?

Even though, it’s very silly, I’m gonna keep using it. At least for now. Tests have been written. I’m not keen on rewriting them. Sigh.

@falsifian@www.falsifian.org In my opinion it was a mistake that we defined the first url field in the feed to define the URL for hashing. It should have been the last encountered one. Then, assuming append-style feeds, you could override the old URL with a new one from a certain point on:

# url = https://example.com/alias/txtxt.txt
# url = https://example.com/initial/twtxt.txt
<message 1 uses the initial URL>
<message 2 uses the initial URL, too>
# url = https://example.com/new/twtxt.txt
<message 3 uses the new URL>
# url = https://example.com/brand-new/twtxt.txt
<message 4 uses the brand new URL>

In theory, the same could be done for prepend-style feeds. They do exist, I’ve come around them. The parser would just have to calculate the hashes afterwards and not immediately.

@aelaraji@aelaraji.com Just move to Mars to get an extra hour a day: https://spaceplace.nasa.gov/days/en/ If that’s not enough, Mercury should have you covered for sure.

@movq@www.uninformativ.de It sounds complicated. After reading it only twice, I haven’t gotten it. :-D

Yes, I’m all for dedicated message IDs. That would be a whole new format then. But I would be fine with it. The only thing is that all our clients have to be touched. At the moment, I do not worry about spoofing (however, I definitely should).

Fuck, I lost my pocket knive somewhere.

When we passed a few horses in the forest, there was really strong soup odor in the air. It didn’t smell like horse at all, but soup. Maybe they’ve been soup horses, chickens were out of stock.

29°C, zero wind, extremely humid, luckily the sun was behind the clouds. I’m soaking wet, sweat ran down in streams and dripped in my eyes, it burned a bit. The sky is getting a little dark, I hope the thunderstorm and rain are really arriving here later. Rain had always been finally cancelled the couple last days.

I’m gotta go cool off my fingers now, they’re swollen from the heat.

My honey bread fell on the floor and would you believe which side was down? The clean, not the coated one. Witchcraft!

I’m out of shape. I decided to walk up the local mountain to watch the sunset, but I arrived five minutes late, even though I sped up at the end. Should have started my journey ten or fifteen minutes earlier. I saw the setting sun at foot, but the photos were total disasters.

On the way there I picked two handful of blackberries in the forest. Delicious!

Today was the second time in my life that I saw a grass snake in the wild. They can easily be recognized by the yellow “ears”. Unfortunately, this one was run over. :-( But I jumped at the opportunity to photograph it as it didn’t escape in a fraction of a second like my first encounter three years ago. Still, poor fellow. :-(

Image

On the way home, a deer jumped out of the brush in front of me and headed down the forest road before it went back in the other side. As always, that’s nice.

I also had to slow down a bunch of times because of frogs or toads on the paths. Not sure which ones, it was already after dark. I guesstimate it must have been 60-70 amphibians in total, maybe more. Some of them did not move to the wayside but rather into the middle of the track, right in front of me. Crazy suicide frogs! There were four reeeeaaaallly close calls. I could just avoid stepping on them after they tried to hop right under my boot. Not a centimeter to spare. No toads were harmed during my trip. Phew!

Once I had to stop completely because of the large activity ahead of me. A larger (about the size of half a palm) individual surrounded my foot and then jumped against my heel. Twice! What the heck!? :-D But suuuper cool experience. I’m very glad I actually went out. Totally worth it. I met so many amazing animals. Don’t care about the missed sunset a single bit.

Yeah, user error on my end, never mind. The persisted settings.yaml overrides the command line arguments. That’s surprising to me. I expected the command line options to overrule the config file. Oh well.

@abucci@anthony.buc.ci You can also use -R=false on the command line or leave it out entirely. When explicitly stating -R=false, there has to be an equal sign. With a space (-R false) it’s somehow parsed as -R which is equivalent to -R=true. O_o Very weird. I’d really like to see an error instead.

I still have to figure out the precedence of the settings.yaml or command line arguments. I’m probably holding it wrong, but it seems to give me different results…

@abucci@anthony.buc.ci Thank you for using Lyse’s Unofficial Yarnd Help Desk: https://lyse.isobeef.org/tmp/yarnd-disable-registrations.png

It cooled off to 20°C today, but mid week is supposed to be crazy hot again. It was a nice walk, also plenty of people around, though. So we decided against going up our backyard mountain to avoid the masses. We finally took a path that we haven’t checked out for years. That was pretty cool. I couldn’t remember anything on that.

Image

More scenery: https://lyse.isobeef.org/waldspaziergang-2024-08-25/

Neither of us has ever seen such a marmelade bun mushroom:

Image

@falsifian@www.falsifian.org @bender@twtxt.net I’d certainly hate my client for automatic feed unsubscription, too.

Hmm, bissel kalt… https://lyse.isobeef.org/tmp/null-grad.png

@prologic@twtxt.net Yeah, I’ve noticed that as well when I hacked around. That’s a very good addition, ta! :-)

Getting to this view felt suprisingly difficult, though. I always expected my feeds I follow in the “Feeds” tab. You won’t believe how many times I clicked on “Feeds” yesterday evening. :-D Adding at least a link to my following list on the “Feeds” page would help my learning resistence. But that’s something different.

Also, turns out that “My Feeds” is the list of feeds that I author myself, not the ones I have subscribed to. The naming is alright, I can see that it makes sense. It just was an initial surprise that came up.

Righto, I cobbled something together here: https://git.mills.io/yarnsocial/yarn/pulls/1172 It needs a bunch more work, though. Screen time is up for today.

I’m wrong! Both 404 and 410, among others, are considered dead feeds: https://git.mills.io/yarnsocial/yarn/src/branch/main/internal/cache.go#L1343 Whatever that actually means.

@bender@twtxt.net You could be right. Grepping the yarnd code for 410 and Gone did not reveal anything. Maybe, maybe it is handled by another library. But I kinda doubt it.

@bender@twtxt.net 404 could be indeed a temporary error if the file resides on a mounted remote filesystem and then the mount point fails for some reason. With a symlink from the web root to the file on the mount, the web server probably will not recognize the mount point failure as such. Thus, it might not reply with a 503 Service Unavailable (or something like that), but 404 Not Found instead. (I could be wrong on that, though.)

The right™ way is to signal 410 Gone if the feed does not exist anymore and will not come back to life again. But that’s hard to come by in the wild. Somebody has to manually configure that in almost all situations.

But yes, as @falsifian@www.falsifian.org points out, exponential backoff looks like a good strategy. Probably even report a failure to users somehow, so they can check and potentially unsubscribe.

My mate just said “WTF, there’s somehow a wall lizard in the family tree” and I can only agree: https://www.youtube.com/watch?v=ezgweVd4a98 Looks like this is the first real evidence of those lizard people. :-D

Transformed four kilograms of blackberries into a bit over three kilograms of blackberry jelly. https://lyse.isobeef.org/brombeergelee-2024-08-19/ The leftover jelly did not fit in prepared canning jars, so I dumped it in a regular drinking glass (which was a mustard glass in its former life):

Image

I saw a kestrel on a power pole the other day. It then flew off and attacked another one sitting in a tree:

Image

More peaceful before that: https://lyse.isobeef.org/turmfalke-2024-08-07/

@bender@twtxt.net Haha, no worries. I do like that you enjoyed your real life and not wasted it online. :-)

But I’m wondering how you discovered it a week later. Are you somehow regularly checking complete recent feed histories?

Heck yeah, I already saw three shooting stars in 10 minutes. Perseids are awesome! :-)

Happy 28th birthday, @bender@twtxt.net! 8-)

@stigatle@yarn.stigatle.no Enjoy the time off! I reckon I should also plan for some vacation in the near future.

@prologic@twtxt.net @xuu@txt.sour.is Hmm, looks like this fix was based on my unsuccessful attempt to repair it, hence heaps of tests are failing when I run make test. I will look into that.

And now, @bender@twtxt.net’s feed changed, too. Bracketed text got duplicated once again.

How do the feeds look on disk? Do they already contain this bracketed text?

For reference, I just placed a copy of the feed here: https://lyse.isobeef.org/tmp/bracketed-text/bender-2024-08-04-10-34.txt

I haven’t marked the changed twts by @mckinley@twtxt.net as read last time, so I don’t know if something changed there as well. In any case, current snapshot: https://lyse.isobeef.org/tmp/bracketed-text/mckinley-2024-08-04-10-39.txt

Yarnd gossipping might be the reason for the truncated stuff, @prologic@twtxt.net. Who are your peers? Any obvious broken yarnd version or even some kind of an attacker involved? But there must be something else broken in yarnd for the bracketed text to be duplicated.

Paid the forest recreation week a quick visit and checked out their huts they built this week. My camera sucked hard, most photos turned out really crappy. Oh well. https://lyse.isobeef.org/waldspaziergang-2024-08-02/

Image

@stigatle@yarn.stigatle.no Congrats, mate! That’s a funny story. :-) Enjoy your day of glory!

Oh, the lovely smell of rain. Wonderful! 33°C today, yikes! And I can already hear the thunder rolling in.

@thecanine@twtxt.net Step 1: Uninstall it. Step 2: Success! :-D

@prologic@twtxt.net I will try. Which time was it again?

The search engine is broken:

Error error parsing created field: parsing time “1689093798000000000” as “2006-01-02T15:04:05Z07:00”: cannot parse “093798000000000” as “-”

Went for a walk onto my backyard mountain again and ate the first three wild blackberries of the season. Watching the sunset unfold from the summit was quite spectacular. The solar disk was glowing extremely blood red. The photos show it way too white, though.

Image

More: https://lyse.isobeef.org/waldspaziergang-2024-07-24/

I just heard AC/DC play live in Stuttgart-Bad Cannstatt for the first time in my life!

Well, when I was waiting for my train home on the station platform. I didn’t recognize that it was Acca Dacca, and I tried hard. In fact, the stage was 500 meters away, so it was just some music-like sound that found its way into my ears. Still kinda cool to know that I heard them live.

I had a barbie with my old workmates. That’s why I have a story to tell now. On the way there, the train was hopelessly overcrowded with AC/DC fans. You couldn’t fall down, simply impossible. It was like in the videos of Japanese subways, where guards press in passengers to utilize every square centimeter. At later stations, plenty of people didn’t get in. Not a chance. This caused quite some delays. And boy, was it hot in there. Streams of sweat running down everywhere.

Originally, I wanted to meet up with a workmate in a city train for the second part of the trip. Due to a signal failure, his train was delayed, though. It got delayed even more and more and was finally cancelled altogether. I eventually got my connecting train while he was still stuck and decided to abort mission and go home after 40 minutes. Catching my connection was another adventure. It was rerouted to another platform, of course without announcement. Because why would you? Fuck the passengers! Luckily, I noticed that it took a different branch at the switch on arrival and ran down and up the stairs to the other platform. The delay counter in this train showed 40 minutes when I finally got off.

With the exception of Acca Dacca, the way home was pleasantly uneventful. Just a few minutes delay and a relatively low passenger volume.

I’m so grateful for not having to experience all this shit on a daily basis anymore. Not looking forward to the next time I have to go into the city. Not at all.

@movq@www.uninformativ.de @prologic@twtxt.net Just slowly creating software is not enough. It doesn’t cut it. The focus has to be on quality. I see suuuper slow progress in a project, but the quality still totally sucks balls.

Some colored spots in the sky this evening: https://lyse.isobeef.org/abendhimmel-2024-07-14/

Image

My mate and I went on a hike yesterday: https://lyse.isobeef.org/waldspaziergang-2024-07-13/

Image