Notices by Allison Parrish (aparrish@mastodon.social), page 10

enumerating a project's problems and shortcomings in the issue tracker as a way of Just Moving On To The Next Thing For The Love Of God

okay I just pushed an update and uploaded a new archive with (more) correct character encoding—enjoy https://github.com/aparrish/gutenberg-dammit

@jleedev yeah I think this is probably what happened!

@darius that would be really weird in this case I think—this is the file it came from http://www.gutenberg.org/ebooks/2090

@darius wouldn't that imply that the text "Updater's note: This file has been recoded to UTF8" was in the *physical* copy?

actually I love this hypothetical conlang that is English where the dative case is marked by putting an acute accent over the last letter of a word and adding -s

found in a project gutenberg file: "Updateŕs note: This file has been recoded to UTF8." (yes, that is LATIN SMALL LETTER R WITH ACUTE)

I can't tell if this is supposed to be a joke or not

@jfriedhoff mastodon has always been like this (scroll back through my feed and look at literally any post about computer programming). not sure what to do about it. the main benefit is that on mastodon you only get a handful of toots about whatever topic you were talking about instead of threads that go on and on for days where the most important person in the world involved in that topic gets tagged and you get notifications about the whole thing weeks and weeks later

also lots of files (>1%?) that say "yeah I'm utf8 sure whatever" but are actually ISO-8859-1 (according to chardet at least)

lessons learned: (a) never trust someone's claim about the encoding of a text file (b) character encodings are bad and trying to digitize text in the first place was bad idea

if you were going to download this today, maybe hold off—I found a frustrating bug where some utf8-encoded texts were being decoded incorrectly with a different encoding, leading to hilarious mojibake when they came out the other end—will post a fix in a few hrs

none of this would be a problem if the reported charset in the metadata was always the correct charset. but there are a lot of texts that report "us-ascii" when what they really mean is "ascii with occasional 8-bit chars just for fun!"

@hackerfriendly that's next on my to-play list actually!

🎵 people... who boost posts to reach more people... are the luckiest people in the world 🎵