Jonkman Microblog
  • Login
Show Navigation
  • Public

    • Public
    • Network
    • Groups
    • Popular
    • People

Notices by Nolan (nolan@toot.cafe), page 9

  1. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:25:09 EDT Nolan Nolan

    Context is this toot, which I dunno if it's still accurate since it was 3 months ago. But you can see the _resizeComponent() call in there, even if it's not the majority of the trace. https://nulled.red/@flussence/102122451948261638

    In conversation Thursday, 12-Sep-2019 16:25:09 EDT from toot.cafe permalink

    Attachments

    1. File without filename could not get a thumbnail source.
      New status by flussence
      By flussence 🦖️‍🍍️ from nulled.red
  2. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:22:53 EDT Nolan Nolan
    • connor

    @notwa LineageOS doesn't wipe after an update, correct. Sometimes though you may need to update the firmware. In my experience though you can just install the OxygenOS update zip immediately followed by the LineageOS update zip, then it works.

    In conversation Thursday, 12-Sep-2019 16:22:53 EDT from toot.cafe permalink
  3. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:21:41 EDT Nolan Nolan

    In theory you can fix this by using a <div> instead of a <textarea>. Then you can use CSS to actually make it fit its content. Some web-based text boxes use this solution.

    The problem is that this comes with huge accessibility problems - now you need to make the <div> act exactly like a <textarea> in every way, including for screen readers, copy-paste input, etc. Theoretically it's possible, but it would be a ton of work to get everything right.

    In conversation Thursday, 12-Sep-2019 16:21:41 EDT from toot.cafe permalink
  4. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:19:04 EDT Nolan Nolan

    Part of the reason there is input lag in both Mastodon and Pinafore is because of the auto-resizing textbox. Believe it or not, there is no way in HTML/CSS to make a <textarea> that automatically expands or contracts to fit the content. (Believe me, I've looked.)

    The best solutions are all JavaScript-based, which requires recalculating layout every time a character changes (e.g. you press a key). In Pinafore I mitigate this by syncing layouts to frames, but it's not a perfect solution.

    In conversation Thursday, 12-Sep-2019 16:19:04 EDT from toot.cafe permalink
  5. David Chartier (chartier@toot.cafe)'s status on Thursday, 12-Sep-2019 14:12:21 EDT David Chartier David Chartier

    Moved over my growing collection of #Mastodon + #Fediverse tools, tips, videos, etc. bookmarks to Raindrop too: https://raindrop.io/collection/8245323

    Please share any good'uns you think should be in here. I definitely want to grow this.

    In conversation Thursday, 12-Sep-2019 14:12:21 EDT from toot.cafe permalink Repeated by nolan

    Attachments

    1. Raindrop.io - Your smart bookmarks
      A beautiful way to save the most important
  6. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:05:42 EDT Nolan Nolan

    My only major critique of LineageOS is that I still think it's largely inaccessible to non-techies. But it definitely works well for me as my daily driver (on a OnePlus 5T).

    My previous blog posts on the subject:
    - serious: https://nolanlawson.com/2017/11/27/living-with-an-open-source-phone/
    - satirical: https://nolanlawson.com/2019/06/13/how-to-de-google-your-android-phone/

    In conversation Thursday, 12-Sep-2019 16:05:42 EDT from toot.cafe permalink

    Attachments

    1. Living with an open-source phone
      By Nolan Lawson from Read the Tea Leaves

      A few months ago, I decided to make a radical change to my smartphone lifestyle. I wanted a phone that would limit the amount of personal data I was broadcasting to third parties – namely Apple or Google – and decided to run a more-or-less vanilla version of Android, without any Google Play Services (i.e. no Google Maps, GMail, Google Play, or Google Search apps).

      I’ve been living with this setup for a few months, and to my surprise, it’s really not that bad. It doesn’t feel like a return to the Stone Age, nor does it feel like I’ve sacrificed all the niceties of a smartphone to the point where I’m carrying around a glorified flip phone.

      However, it’s a bit non-obvious how to get all of this stuff to work, and especially how to get by with a Android phone that doesn’t have the Google Play Store. So in this post, I’d like to talk a bit about my smartphone setup, in the hope that it might help someone who’s looking to make a similar switch.

      Choosing an Android ROM

      There are various flavors of Android out there: you can choose LineageOS (the spiritual successor to CyanogenMod), Paranoid Android, CopperheadOS, or you can even just build AOSP (the Android Open Source Project) from source.

      After fumbling around with building AOSP from source (which was much more difficult than I thought – lots of requirements for the build environment, CPU horsepower, and storage), I also tried LineageOS and CopperheadOS, and eventually settled on LineageOS.

      Why LineageOS? Well, mostly because it was easy, it offered fast security updates, and I like the minimalist interface and built-in apps. LineageOS was also familiar to me, as I had previously used Cyanogen for many years. (I forgot how much I enjoyed the small creature comforts, such as long-pressing the volume up/down keys to skip tracks!)

      Default homescreen on LineageOS

      I also needed a recovery image in order to install the ROM, and it seems that TWRP has become the de-facto standard these days, replacing the venerated ClockworkMod.

      As it turns out, none of these options will result in a 100% open-source phone, as you’ll still be running the vendor binaries for your particular phone. But this is about as close as you can get to a smartphone that runs only free/libre open-source software.

      Choosing a phone and installing a ROM

      This isn’t going to be a guide to unlocking or rooting your Android phone. The process is different for every phone, and it would take too long to describe all the various steps. Suffice it so say that it’s still a painful process, and your choice of phone can either make it easier or harder.

      I’ve been rooting, unlocking, and tinkering with Android phones for a long time, since the days of the HTC Dream and HTC Magic around 2009/2010. Since then, I’ve worked as an Android and mobile web developer, and I’ve become very comfortable with tools like the Android SDK, adb, fastboot, and Android Studio. And yet, to this day I still find installing custom ROMs to be a frustrating and time-consuming experience. I wish it were easier for casual folks to do this stuff.

      The Nexus line of phones (now Pixel) have always been the easiest to customize, and for many years I stuck with the tried-and-true Nexus 5, which is still a surprisingly capable phone despite having been released in 2013. (I swear it ran faster than my 5X, which also kicked the bucket after only a year or two of use.) However, my aging Nexus 5 just couldn’t cut the mustard anymore due to hardware issues (the USB port was too loose; it had become tricky to charge it), and so I decided to buy a new phone instead.

      I settled on the Samsung Galaxy S5, mostly because I could get it for cheap (<$200 on Amazon) and with full support for my carrier (T-Mobile). In retrospect, choosing a non-Nexus device made my life a lot harder, and after several hours of research on unlocking Samsung phones (including building Heimdall from source on a Windows machine, because the prebuilt binaries were out of date but the software was too old to build on a Mac), I finally had my LineageOS phone up and running.

      The second thing you'll want to do is ensure that your device is encrypted, which you can enable in the security settings. Unfortunately this ended up making my phone unable to start the OS, but after booting into recovery and doing a factory reset, I had both encryption and the OS up and running just fine.

      My LineageOS home screen

      Getting apps

      Without the Google Play Store, you'll have to use F-Droid, which has the added benefit of only hosting free and open-source apps. In fact, a couple of my own apps were previously on there (Catlog and KeepScore), and as far as I can tell, they were built directly from source. (I’m not sure why they were removed; possibly because I stopped maintaining them.)

      To get F-Droid, you just download it directly from f-droid.org. You'll also have to allow installation from "unknown sources" in the security settings.

      For the odd app that isn't available on F-Droid, you can also use the Yalp Store, which can either use your Google account or an ephemeral account to download apps from the Play Store. It’s not clear to me whether or not this violates the Google Play Terms of Service, though, so proceed with caution.

      Messaging

      I use Signal as my default SMS app and for most messaging with family and friends. Unfortunately it isn't available on F-Droid, but you can download it directly from the Signal website, or use the Yalp Store technique above.

      Like most Android apps, Signal normally uses Firebase Cloud Messaging (FCM, formerly GCM) to send push notifications. If you don't have Google Play Services installed, Signal will offer to switch to a less battery-efficient background polling mechanism.

      I've used this mechanism for months and only had occasional problems where a large backlog of old messages was suddenly delivered all at once. It also hasn't had a visible impact on battery life. (I usually end the day with half a charge left; the GS5 has a great battery!)

      Web browser

      For a web browser, I mainly use Firefox via the FFUpdater tool, which is clunky but gets the job done. I’ve set DuckDuckGo as my default search engine, and I have uBlock Origin installed as an add-on, which undoubtedly makes my browsing faster and easier on the battery.

      Occasionally though, I do find sites that don’t run so will with mobile Firefox, and for that I use Auto Updater for Chromium, which automatically installs Chromium and updates it. This is a bit nicer than FFUpdater because it does its work silently in the background, rather than requiring you to manually check for updates.

      The built-in browser doesn’t support “Add to Homescreen,” so I find it fairly useless.

      Maps

      I use OsmAnd~, which is nice because it allows you to download maps in advance for offline use. Unfortunately it’s still nowhere near as feature-complete as Google Maps, so if you’re looking for something to help you navigate your car, you may be out of luck. (Update: actually, OsmAnd~ supports turn-by-turn navigation.)

      I also frequently use maps.google.com in Firefox, which works surprisingly well. Besides some minor performance issues, it’s fairly indistinguishable from the native app for basic directions, bus timetables, and store lookups.

      Google Maps running in Firefox

      Living in Seattle, one of my favorite apps is also OneBusAway, which provides up-to-date arrival times for Seattle buses. Unfortunately this app requires Google Play Services in order to work, so I’ve had to do without.

      Weather

      Speaking of Seattle, it’s also important for me to know how much it’s going to rain in a given day. (Spoiler alert: a lot.)

      This happens to be one of those things about LineageOS that’s a bit non-intuitive. To get weather to work, you have to go to the “extras” page and download a weather provider. (In my case, I’m using OpenWeatherMap.)

      Unfortunately, though, even after installing the weather provider, I couldn’t get the built-in “cLock” widget to show the weather. (It keeps saying “network geolocation is disabled,” even though it’s not.) So I ended up installing Forecast widgets, which gets its data from the National Weather Service rather than the built-in weather provider.

      Maybe this is just some bug with my installation or with the hardware itself, but in any case I’m satisfied with this workaround. The Forecast widget looks and works fine.

      Weather and time widget on my homescreen

      Keyboard

      The only non-standard features I really want from a keyboard are 1) emoji support, and 2) swipe input.

      Unfortunately I couldn’t find any open-source keyboard that can do both of these things, and the AOSP keyboard doesn’t seem to support either. The closest I found was AnySoftKeyboard, which at least has an emoji screen. It doesn’t allow you to search for emoji, though, which is a bit frustrating.

      Sending an emoji via AnySoftKeyboard and Signal

      Overall, the keyboard experience has been my least favorite part of the LineageOS experience. I didn’t realize how much I had become accustomed to swipe input until I had to revert back to tapping, which feels to me like hunt-and-peck. Input is extremely laborious and slow, although on the plus side I’m spending less time texting, so maybe that’s a good thing.

      Update: actually, AnySoftKeyboard does support searching emoji! You just need to type : and then the search term.

      Music and podcasting

      I’m a fairly old-school music consumer. I prefer to listen to albums from start to finish, and I never really got into services like Pandora or Spotify. Instead, I buy my MP3s from Amazon (old habit, my entire library is there) and then sync them from my desktop computer to my phone using adb-sync.

      For browsing and listening to my music library, I’ve found Vanilla Music to be fantastic. It has a cover art downloader, the interface is minimal and clean, and it can play an album from start to finish, which is all I really ask.

      Vanilla Music with some of my albums

      Oh, and of course Vanilla Music sets the cover art on the lock screen. It’s the little things that count.

      I’m also something of a podcast addict, so I use AntennaPod to download and listen to podcasts. I’ve tried several podcast apps on F-Droid, and I found this one to be the easiest and most reliable overall. I particularly like that it allows me to search on iTunes, since some podcasts can be hard to find elsewhere.

      Social media

      I stopped using Twitter, but if you absolutely must, their mobile website is not bad in either Firefox or Chromium. I’m told Facebook’s mobile website is also pretty functional.

      I spend most of my social media time these days on Mastodon, and as it turns out the mobile websites work perfectly fine in both Firefox and Chrome, so you can just pin it to your homescreen. There’s also Tusky if you prefer the native app experience.

      Ridesharing

      I don’t use Uber, but Lyft has a mobile webapp that works just as well as their native app. So when I need a ride, I just open ride.lyft.com in Firefox. It’s a PWA, and as far as I can tell it works just as well as the native Android app.

      Somewhat bafflingly, though, if you go to lyft.com directly, the site may try to route you to their driver portal or to the app store instead.

      Lyft’s website may offer “Lyft in Firefox,” which is the PWA

      In case you get confused, you may need to look for the “Lyft in Firefox” link. To me, this seems like a bizarre way of saying “Lyft for the web,” but you do you, Lyft.

      Email, calendar, and contacts

      I switched to FastMail recently for my primary email account, and I’ve been really happy with the service. It’s fast, it’s simple, and it has a clear business model where I pay them 5 bucks a month and they host my email. Simple.

      To get my email on my phone, I use K-9 Mail, as I’ve found it a bit more reliable and feature-rich than the built-in Email app. If you prefer a simpler interface, though, the built-in app works fine too.

      FastMail also supports calendar and contact sync, and this is where it gets a bit trickier. If you’ve ever connected an email client to your email server by manually typing the IMAP and SMTP settings (which, incidentally, you’ll have to do for K-9 email), then it’ll feel pretty familiar. FastMail supports CalDav and CardDav, and so to hook these up to my phone I used DavDroid.

      Honestly this was probably the most tedious process of setting up my LineageOS phone. It involved creating app passwords for each client (I use separate passwords for email, calendar, and contacts, in the interests of security), and manually typing in the server names and ports for the various FastMail endpoints. This was a long and error-prone process, but in the end I do have full email, calendar, and contact sync, so I can’t complain.

      Passwords and two-factor auth

      For nearly a decade I’ve stored my passwords using the Joel Spolsky method, which is to use PasswordGorilla on desktop and PasswdSafe on Android. I like this method because it’s simple, it works cross-platform, and I maintain control of the password file.

      For two-factor authentication (other than basic SMS), there’s FreeOTP, which essentially takes the place of Google Authenticator.

      The web works for everyone

      One of my main reasons for switching to a non-Google Android phone was to see how capable the web is as an application delivery system. And aside from the native apps listed above and some minor utilities (e.g. a barcode scanner and a notes widget), I do pretty much everything in a web browser.

      Media sources like Hacker News, Ars Technica, and others don’t need an app – you can just pin a website to your home screen. (Although for Hacker News, I use the excellent hn.premii.com.) The main exception for me is NewsBlur, which I access via the open-source client app.

      For videos, both YouTube and Vimeo also work great as mobile webapps. In Firefox, they can run both in fullscreen mode and in the background.

      For file sync, I use Dropbox’s mobile webapp for quick file downloads from my Dropbox account and Firefox Send for sending more ephemeral files. I need a better solution to backing up photos, though; for now I’m just using adb-sync to sync to my Dropbox folder on the desktop.

      I also do work at Microsoft, and although I have a personal policy of not syncing my work email to my phone, I can make a temporary exception by loading outlook.office.com in Firefox, which works surprisingly well. For those odd moments when I need to send an email or check the location of a meeting, it gets the job done.

      As a guitarist, I also needed a way to tune my guitar, and sadly the excellent g-strings is both paid and closed source, so it was a no-go. So I use Paul Lewis’ guitar tuning webapp instead, which is a good substitute.

      Me singing a bit off-key into a guitar tuner

      Turns out the web of 2017 is capable of quite a bit, from hailing a taxi to tuning your guitar!

      Conclusion

      I hope I’ve demonstrated in this blog post that it’s not only possible to use an open-source phone without any of the pervasive tracking, spyware, or bloatware that we’ve come to expect from most smartphones, but it’s actually quite viable and even enjoyable.

      When I booted up my Galaxy S5 for the first time, I was immediately greeted with a barrage of ads for Samsung and T-Mobile services and upsells, all in the 5 minutes it took to turn on USB debugging so I could install custom software and mercifully silence all of the nagware. My clean and stripped-down LineageOS setup is about as far from that horrendous out-of-the-box experience as you can get.

      Note though, that this is the setup that works for me, and your mileage may vary depending on how much you rely on various software and services. If you use GMail and GDocs, you may be better off just using a Google-flavored Android phone. And if you’re bought into the iTunes and iCloud ecosystem, it may be more trouble than it’s worth to switch from an iPhone.

      Also, despite my attempts to de-mystify some of the less obvious parts of this setup, I’m in no way claiming that any of this is accessible to the non-geek, non-hacker crowd. The mere process of installing LineageOS is so far beyond the capabilities of the average non-techie that it actually fills me with a kind of despair.

      This is probably a subject for an entire post, but I’m becoming concerned that my friends and I in the open-source, pro-privacy hacker scene are building a two-tiered world, where the tech elite can successfully install and configure software that maintains their security and privacy, whereas the average person is stuck either paying a premium for a privacy-respecting but closed-source iPhone, or paying a reasonable price for an Android phone where their security and privacy are far from assured.

      That said, maybe if more of us inhabit this (admittedly neophyte-unfriendly) world, then maybe we can work to make it more accessible to those of us who don’t know how to use a command line and have no patience to type out IMAP server URLs into their smartphone. Let’s hope so.

    2. File without filename could not get a thumbnail source.
      How to de-Google your Android phone
      By Nolan Lawson from Read the Tea Leaves

      First, download a ROM from this Russian message board. It’s okay! You can totally verify the GPG signature. Allow yourself 30 minutes to remember how GPG works, then verify that forum poster LeetAndrej420 has indeed signed the file.

      Next, root your Android phone. You will need to hold the volume-up and power buttons for ten seconds, then unplug from USB, then reboot a few times after you mess it up, then give up and download the Android dev tools.

      After you figure out the Android adb and fastboot commands, you should see a friendly UI with green Courier text on a black background. Press the button that says, “I void my warranty and completely exonerate the OEM in the likely event that I am actually pwning myself by installing random software from the internet onto a tracking device I carry in my pocket every day.” But it’s okay. You trust Andrej, right?

      Next you will need to install the “recovery” tool. Despite the name, this is actually the best way to brick your device. Luckily it is incredibly feature-rich, boasting 12 buttons on the home screen, including an “Advanced” button containing more buttons. These buttons will invite you to do things like “clear the Dalvik/ART cache,” which you totally know what that means.

      When you download the recovery tool, make sure you get the right version for your phone! Of course, it’s not named after your phone’s brand name, but rather a cheeky internal name chosen by the OEM, like “bacon”, “cheeseburger”, or “mahimahi”. The professionalism on display from all parties should fill you with confidence.

      You will download the recovery tool from a site called SickWarez.biz. Use GPG to ensure that it’s signed by Andrej.

      Once downloaded, go into recovery mode and install the ROM, being careful to press the one correct button out of 12, like a game of Minesweeper that will brick your phone if you lose. This will also factory-reset your device, which is fine because all your photos and contacts are backed up to your Google account… ah, right. You’ll want to do something about that.

      Assuming you have successfully installed the ROM without turning your phone into a $700 doorstop, you can now install apps. Thankfully there is F-Droid, which hosts all your favorite open-source apps. Wait, your favorite apps aren’t open-source? Well, at least it has Signal. Wait, it doesn’t have Signal?

      Once you’ve installed the Yalp Store, which sideloads apps from Google Play in a way that may or may not be totally illegal and will get blocked by Google once they read this blog post and realize that it exists, you can now download some actually useful apps.

      Thankfully, though, your personal data will be safe and secure from third-party developers, because these apps will not work. Be prepared for error messages like, “Please install Google Maps,” “Google Play Services required,” or “What kind of sicko has a Google phone without Google? What is wrong with you?”

      After all this ceremony, you can now relax and enjoy your Google-free Android device. Note, though, that weather widgets, GPS, push notifications, and the majority of Android apps you rely on will not work. That said, there are some great note-taking apps! Plus SMS will still work. Good old SMS.

      So now that you’ve successfully turned your $700 Android device into a glorified $30 Nokia flip phone, which may or may not be siphoning your passwords to a Ukrainian teenager, you can finally have a Google-free smartphone experience. Or you could just buy an iPhone.

  7. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 16:00:48 EDT Nolan Nolan

    Just became a patron of LineageOS. They do a good job, and somebody has to pay for the servers. https://www.patreon.com/LineageOS/overview

    In conversation Thursday, 12-Sep-2019 16:00:48 EDT from toot.cafe permalink

    Attachments

    1. File without filename could not get a thumbnail source.
      LineageOS is creating a free and open source operating system based on Android | Patreon
      from Patreon
      Become a patron of LineageOS today: Read posts by LineageOS and get access to exclusive content and experiences on the world’s largest membership platform for artists and creators.
  8. mhoye (mhoye@mastodon.social)'s status on Thursday, 12-Sep-2019 15:40:58 EDT mhoye mhoye

    https://discourse.mozilla.org/t/synchronous-messaging-at-mozilla-trial-servers-feedback/44871

    In conversation Thursday, 12-Sep-2019 15:40:58 EDT from mastodon.social permalink Repeated by nolan

    Attachments

    1. File without filename could not get a thumbnail source.
      Synchronous Messaging at Mozilla - Trial Servers & Feedback
      from Mozilla Discourse
      Hi, everyone - The blog post is here: http://exple.tive.org/blarg/2019/09/06/forward-motion/ … but I wanted to make sure everyone here had a chance to see it. I’m happy that I’ve been able to announce our list of finalist candidate stacks. Once we’ve got them spun up - later this week, if all goes well - we’d be grateful for your feedback. I’ll have docs available with the URLs as soon as we’re ready to go live, and I’ll be setting up feedback channels on each of the servers, as well as a thr...
  9. SoapDog ✓✓✓ 3 times verified! (soapdog@toot.cafe)'s status on Thursday, 12-Sep-2019 15:20:14 EDT SoapDog ✓✓✓ 3 times verified! SoapDog ✓✓✓ 3 times verified!

    Today I'm feeling quite nostalgic about #FirefoxOS:

    https://andregarzia.com/2019/09/nostalgia-for-firefox-os.html

    In conversation Thursday, 12-Sep-2019 15:20:14 EDT from toot.cafe permalink Repeated by nolan
  10. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 10:47:54 EDT Nolan Nolan

    And of course, Pinafore has a "wellness" setting where you can just turn off boost/fav/follower counts entirely. I don't personally use this feature, though.

    In conversation Thursday, 12-Sep-2019 10:47:54 EDT from toot.cafe permalink
  11. Nolan (nolan@toot.cafe)'s status on Thursday, 12-Sep-2019 10:46:38 EDT Nolan Nolan

    "Would the Internet Be Healthier Without 'Like' Counts?" by Paris Martineau https://www.wired.com/story/internet-healthier-without-like-counts/

    I've always liked Mastodon's design where the likes/boosts aren't shown until you click a toot. It helps me focus on reading the content without immediately trying to gauge its social approval.

    In conversation Thursday, 12-Sep-2019 10:46:38 EDT from toot.cafe permalink
  12. Nolan (nolan@toot.cafe)'s status on Wednesday, 11-Sep-2019 21:27:51 EDT Nolan Nolan

    The part about open-source burnout really struck close to home for me. Sometimes I feel like you put something out there that solves your problem, but then you get a bunch of issues/PRs solving someone else's problem, and after a while it's hard to muster the will to care.

    The best projects are indeed those that are tiny and maybe only matter to a small group of people. That's why I like working on Pinafore these days more than databases, libraries, etc.

    In conversation Wednesday, 11-Sep-2019 21:27:51 EDT from toot.cafe permalink
  13. Ë—ËŹË‹ wakest ËŽËŠË— (liaizon@social.wake.st)'s status on Monday, 09-Sep-2019 16:17:32 EDT ˗ˏˋ wakest ˎˊ˗ Ë—ËŹË‹ wakest ËŽËŠË—
    • Darius Kazemi

    somehow I missed that Marketplace did a feature about the #fediverse with @darius!

    https://www.marketplace.org/shows/marketplace-tech/diy-build-your-own-social-media-site-guide/

    In conversation Monday, 09-Sep-2019 16:17:32 EDT from social.wake.st permalink Repeated by nolan
  14. phae (phae@toot.cafe)'s status on Monday, 09-Sep-2019 15:43:19 EDT phae phae

    I see people complaining that Duolingo doesn't teach practical sentences...

    In conversation Monday, 09-Sep-2019 15:43:19 EDT from toot.cafe permalink Repeated by nolan
  15. Nolan (nolan@toot.cafe)'s status on Monday, 09-Sep-2019 22:07:36 EDT Nolan Nolan
    • 𝓻𝓻𝓪

    @rra Thanks for sharing your tips! Cleaning the yarn cache is a helpful one for me (especially since I run multiple instances on one server).

    Also: switching from local media storage to an S3-compatible server was a big help for me. Disk doesn't grow nearly as fast now (just the Postgres database, not the media files).

    I'm also curious how other #MastoAdmins are dealing with the memory growth. I have to restart every month or so, or it reaches my server's max of 8GB (running 3 instances).

    In conversation Monday, 09-Sep-2019 22:07:36 EDT from toot.cafe permalink
  16. 𝓻𝓻𝓪 (rra@post.lurk.org)'s status on Monday, 09-Sep-2019 08:22:26 EDT 𝓻𝓻𝓪 𝓻𝓻𝓪

    I know there are quite some #mastoadmin struggling with the disk space usage of the mastodon instance they maintain. The issue is not acknowledged enough and there are very few ways you can mitigate this using the official mastodon tools etc. Even better, often the proposed 'solution' is to 'get more disk space lol'.

    Here's a compilation of recipes on how we try to deal with this for post.lurk.org as we learn by doing.

    https://things.bleu255.com/runyourown/Mastodon#Maintenance

    In general we document many parts of lurk.org on that wiki so that other collectives can use that information:

    https://things.bleu255.com/runyourown/Main_Page

    In conversation Monday, 09-Sep-2019 08:22:26 EDT from post.lurk.org permalink Repeated by nolan
  17. David Chartier (chartier@toot.cafe)'s status on Monday, 09-Sep-2019 17:27:47 EDT David Chartier David Chartier

    Hey Masto, who are some good general #tech folks to follow around here? I’d like to find more of that convo and community away from bird hell.

    Anyone who spends at least some time on Apple stuff, or mobile/iPad/apps chat? General tech works too.

    In conversation Monday, 09-Sep-2019 17:27:47 EDT from toot.cafe permalink Repeated by nolan
  18. Nolan (nolan@toot.cafe)'s status on Monday, 09-Sep-2019 10:34:28 EDT Nolan Nolan

    For the record, right-clicking an app icon in the taskbar on Ubuntu, on my XPS 13, is basically instant.

    In conversation Monday, 09-Sep-2019 10:34:28 EDT from toot.cafe permalink
  19. Nolan (nolan@toot.cafe)'s status on Monday, 09-Sep-2019 10:33:05 EDT Nolan Nolan

    "Taskbar Latency and Kernel Calls" by Bruce Dawson https://randomascii.wordpress.com/2019/09/08/taskbar-latency-and-kernel-calls/

    Windows Performance Analyzer is actually an amazing tool, and something I miss from my days at Microsoft. Also I'm not surprised to see that right-clicking the taskbar is super slow in Windows. Maybe it's just my bias, but every OS interaction just *feels* slow to me on Windows, which is partly why I switched to Ubuntu.

    In conversation Monday, 09-Sep-2019 10:33:05 EDT from toot.cafe permalink

    Attachments

    1. Windows Live Photo Gallery—Poor Performance Peculiarities
      By brucedawson from Random ASCII - tech blog of Bruce Dawson

      I use Windows Live Photo Gallery (WLPG) because it has powerful features for tagging my photos and sorting through them. I depend on its functionality. But I have long been annoyed by its inconsistent performance, and today I investigated.

      My sad conclusion is that it appears that WLPG does a lot of extraneous I/O when zooming images, then busy waits for the results. The net result is that if you try using WLPG on a laptop without an SSD you will get poor performance and poor battery life. It is possible to work around this, and double the performance of WLPG, but it’s a tricky operation.

      If Microsoft wants Windows to be seen as fluid and fun, like other OSes then it needs to take stuff like this seriously and make sure that avoidable performance problems don’t ruin an otherwise excellent product.

      The setup

      I frequently notice small hangs (from one to five seconds) when tagging photos or browsing through photos with WLPG. I have a four-core/eight-thread Sandybridge machine with 8 GB of RAM and there is, I believe, no reason for significant delays as I skim through my photos. I eventually found a simple scenario that let me reliably produce a short hang, and this is what I analyzed. It is not the worst hang I have seen – those when tagging photos or rapidly advancing through photos are much longer – but it is the simplest, and I suspect it is representative of the architectural problems that make WLPG not as responsive as it should be.

      Zooming in

      Zooming in for a closer look on an image is a pretty basic operation. The image is already decoded so it should be fast. And indeed, once I’ve started zooming in on an image the frame rate is excellent – it keeps up with my mouse wheel. But on the initial roll of the mouse wheel on a new image there is a delay of about a second before the first image is displayed. I turned on my ETW input user-provider (yes, it’s a key logger) so that I could see keyboard and mouse events and then recorded a trace.

      Rich ETW traces with input data, timer frequency, battery information, optional power draw, and more are available by using UIforETW to record traces.

      In the xperfview screenshot below you can see the three diamonds in the Generic Events graph which represent movement of the mouse wheel, and on the CPU usage graph you can see the burst of CPU activity that was triggered.

      WPA presents this information differently but the basic concepts are the same.

      The zoomed image was not displayed until the end of the CPU activity, so by looking at the time along the bottom we can see that this is about a 1.05 second hang (Xperf Wait Analysis discusses other ways of finding hangs in traces).

      CPU consumption, right?

      The graph clearly shows that there was significant CPU usage (12.5% represents one pegged core on my eight-thread system), and the summary table shows about 1.2 s of CPU time consumed during the 1.05 second hang (across multiple threads).

      So. Slam dunk. The CPU was busy doing hugely important CPU stuff in order to prepare for scaling the image.

      Let’s check. Let’s look at the CPU sampling data to see what we find:

      • ~652 ms in VCOMP90.DLL!_vcomp::PartialBarrier1_Poll::Block
      • ~167 ms in ntkrnlmp.exe!IoRemoveIoCompletion
      • ~68 ms in ntkrnlmp.exe!KiPageFault
      • The remainder is miscellaneous functions that don’t add up to much

      Wait a minute! What’s going on here?

      Please don’t busy wait

      PartialBarrier1_Poll::Block sounds like busy waiting. Busy waiting is bad form. It wastes energy, it harms battery life, and in some cases it actually harms performance. Don’t busy wait. Spinning for a very short period of time might make sense, but it looks like _vcomp::PartialBarrier1_Poll::Block, which appears to be part of Microsoft’s OpenMP implementation, is busy waiting for more than half a second over a one second period. It’s name and the sampling data make it clear that it is doing nothing valuable. That makes me sad. In addition to being a waste of electricity, busy waiting means that wait analysis doesn’t work which means it is impossible to find out what the thread is actually waiting on. Double fail.

      Since we can’t scientifically determine what the code is waiting on we’ll have to guess. The only thing I can be certain it isn’t waiting on is some other CPU bound thread, since every other thread on the system is at least 98% idle during this busy wait.

      It’s the I/O

      The ntkrnlmp.exe!IoRemoveIoCompletion suggests that I/O might be what is waited on. ~167 ms of CPU time for I/O is impressively large and suggests that a lot of it is being done.

      Page faults are I/O too

      The page faults are easily investigated by looking at the Hard Faults summary table. These are not page faults from memory pressure (I made sure I had more than 2 GB available when running these tests). The hard faults summary table shows that they are all from the file “C:\Users\Bruced\AppData\Local\Temp\~PI6E15.tmp”. So clearly WLPG is memory mapping this file and faulting it in. But where did it come from?

      Looking at the disk I/O graph we can see that there was no disk I/O until the selected region, which is when I did the zoom. Clearly the zooming is triggering disk I/O, and this was easily proven through repeated testing. Here we can see the disk I/O triggered by the mouse wheel activity, with nothing happening before the selected region.

      Looking at the Disk I/O summary table (and confirming with the File I/O summary table) we see this:

      WLPG writes 21.9 MB to this temporary file, then reads back 17.9 MB of it. And it takes about 745 ms to do it.

      The images being viewed were all 5184 by 3456, so the data written to the file is too small to be the decompressed image. It is more than enough for a full 24-bit mipmap chain, which would make sense for zooming. The odd thing is that the region of the image being read is different from what is being written. It seems most likely that WLPG is maintaining a large cache on disk, so the writes are flushing out the previous cache and the reads are reading in the new data. Or something like that. Caching mipmaps seems particularly odd since mipmaps can be regenerated faster than they can be loaded from disk.

      Unfortunately the loading (of the presumably cached mipmaps) is slowed down by the simultaneous writes. A hard disk can only do one thing at a time.

      Browsing images

      After realizing that reads and writes to the %temp% directory were the limiting factor in WLPG performance I did some profiling of photo browsing by advancing through images. During a 22.4 second period where 14 images were viewed I measured 12.7 seconds of disk I/O time. Of this disk I/O, fully 80% of the disk I/O time, and 89% of the bytes read and written was spent on traffic to those temporary files. Looking at the CPU usage it is clear that a lot of the delay in switching between images is due to WLPG waiting on this disk I/O, although in this case at least it avoided busy waiting. It seems likely that image browsing performance could be doubled if the traffic to the %temp% directory was eliminated.

      The disk I/O time is particularly frustrating because when you are scanning through images in order, WLPG spends about half of its time writing to these temporary files, and never reading from them. Thus, this significant performance cost has no benefit to this common scenario.

      Subverting the design for fun and profit

      The WLPG designers obviously felt that caching lots of data on disk would improve performance. My tests suggest otherwise. Since I can’t disable the caching scheme entirely I decided to rework it by putting the %temp% and %tmp% directories on a 1.5 GB RAM disk. This is a delicate operation because if I consume too much RAM then Windows will start paging and I’ll replace one disk I/O problem with another, but with an 8 GB system there is room for this to work, as long as I am careful.

      It worked, quite brilliantly. The disk I/O time went down dramatically (100 ms for 700 MB of writes, instead of 7,400 ms for 545 MB of writes), and the browsing performance roughly doubled. The remaining delays appear to be from the actual decoding of the images, which is as it should be.

      That’s all folks

      Due to a lack of symbols in some components (imaging.dll, I’m looking at you) and a lack of source code and architectural documentation I can’t tell exactly what’s going on, but here’s what the trace seems to be telling us.

      • WLPG writes some data to a temporary file. The type of data is unknown but is assumed to be mipmaps, probably from the previously viewed image
      • WLPG  reads from a different portion of the temporary file, through a memory map – perhaps loading previously cached mipmaps?
      • During most of this time WLPG is busy waiting for the I/O to be done
      • For some reason the Windows disk cache, which usually caches all recently written data, doesn’t help
      • Somewhere towards the end of the trace WLPG displays the zoomed image
      • The behavior when browsing images is similar except that the cache is only ever written to

      So that’s where it ends. Maybe the WLPG team can explain in more detail, or maybe they can fix the problem, by not reading and writing ~40 MB of data to disk during a latency sensitive operation.

      Until then I guess I’ll be firing up my RAM disk whenever I have a lot of photo tagging and browsing work to do.

      Update

      The RAM disk works brilliantly. It exposes a few other performance limitations in WLPG (I don’t think they anticipated that somebody would add 33,000 people tags covering over 600 different people) but overall it makes WLPG feel much more fluid.

      Also, I have heard from my unofficial sources at Microsoft that the busy wait in OpenMP is going away in Visual Studio 2012. That won’t actually improve performance (the excessive disk I/O is the real problem) but at least it will avoid the electricity consumption that comes with huge amounts of busy waiting.

      And, in 2014 I finally bought an SSD which makes this problem go away. WLPG could have great performance without an SSD, but it currently doesn’t.

  20. Nolan (nolan@toot.cafe)'s status on Sunday, 08-Sep-2019 22:16:46 EDT Nolan Nolan

    websites: if you click "keep me logged in", it will never work
    ads/trackers: know exactly who you are at all times with zero effort required on your part

    In conversation Sunday, 08-Sep-2019 22:16:46 EDT from toot.cafe permalink
  • After
  • Before
  • Help
  • About
  • FAQ
  • TOS
  • Privacy
  • Source
  • Version
  • Contact

Jonkman Microblog is a social network, courtesy of SOBAC Microcomputer Services. It runs on GNU social, version 1.2.0-beta5, available under the GNU Affero General Public License.

Creative Commons Attribution 3.0 All Jonkman Microblog content and data are available under the Creative Commons Attribution 3.0 license.

Switch to desktop site layout.