Bringing Hometown up to date with mastodon/mastodon

[mistydemeo]

This is a massive PR, so apologies for the poor performance viewing it!

This merges in the content of Mastodon 4.3.0. This PR isn't ready to deploy to production, and I don't recommend trying it in a production environment. Since I started by merging 4.3.0, this will likely have reverted at least some of the content of the post-4.3.0 security patches in the 4.2.x series; those will be brought back as I merge in 4.4.0 and 4.5.0.

I haven't yet tested this PR, so it may not actually run without additional tweaks. I'm putting it up now for discussion and so multiple people can test if we want.

---

Here's a few notes on things I ran into and thoughts I had while working on it:

• We may want to look at server_banner.jsx, where I ended up undoing the v3-style rewrite due to the new react component being used. I'm not sure what the current goal for it is.
• I may not have 100% merged in all content from status_action_bar correctly.
• There's a number of places where I updated the links for remote usernames, and they should be tested to make sure they work. They're all <Permalink> elements. I have in my notes that status_content might need double-checking.
• I may also not have gotten the spoiler display right in status_content.
• It looks like previous versions of Hometown had customizations to the media_item display, not all of which was preserved as-is due to surrounding rewrites. I'm not sure what the vision is and could use another set of eyes.
• The character count rendering clashed with the newly-introduced handling of a character count variable upstream, and I'm not sure if the version I've got here 100% works or not.
• The navigation bar was completely rewritten and no longer has the components Hometown was customizing; did I properly catch any replacements? Did we handle all links to accounts to make sure they go to the remote instance?
• Polls: I reverted the single/multiple choice toggle because upstream has an actual proper UI for this that should be integrated in one of the versions we’re merging in. Please double-check that the current version looks fine to you.
• The font icons will need double-checking. Hometown made some custom use of the font-awesome icons, while Mastodon 4.3.0 replaced font-awesome with something else. I may not have caught all places that font-awesome icons were being used.
• We’ll want to make sure the post display is correct, including Hometown customizations, because the previous static post page has been replaced with the React one.

[mistydemeo]

Merged in Mastodon 4.5.5.

[nachtjasmin]

So, I just performed the task and migrated a running instance from 4.2 to 4.5. 😅

It worked surprisingly well! Kudos! 🥳 The database migrations were all cleanly applied and from what I was able to observe, nothing broke there! The most important change I had to apply were the new Active Record encryption secrets of the 4.3.0 release.

Active Record encryption secrets configuration
Mastodon now requires new environment variables for secret keys to be set.

Generate new secrets and set ACTIVE_RECORD_ENCRYPTION_DETERMINISTIC_KEY, ACTIVE_RECORD_ENCRYPTION_KEY_DERIVATION_SALT, and ACTIVE_RECORD_ENCRYPTION_PRIMARY_KEY accordingly before restarting Mastodon.

After that, I just executed the migration in two steps, similar to the description on the 4.3 release.

jasmin@fedi-2 ~> sudo systemd-run \
                     --uid=mastodon \
                     --working-directory=/var/lib/mastodon \
                     -E RAILS_ENV=production \
                     -E SKIP_POST_DEPLOYMENT_MIGRATIONS=true \
                     -t bundle exec rails db:migrate
jasmin@fedi-2 ~> sudo systemd-run \
                     --uid=mastodon \
                     --working-directory=/var/lib/mastodon \
                     -E RAILS_ENV=production \
                     -t bundle exec rails db:migrate

⚠️ Do not remove the OTP_SECRET ahead of the migrations, it's required to convert the existing values in the database. Without it, the migration will fail, so admins should keep it until they executed the migrations.

All in all, the migrations took maybe 2 to 3 minutes on our 35GB (uncompressed) database.

---

Issues I found

Missing information in locales

Right now, the rendering of several haml files is failing because of:

Information for cause: I18n::MissingInterpolationArgument (missing interpolation argument :title in "%{title} hosted on %{domain}" ({domain: "queer.group"} given))

Because of this, Mastodon was first replying with the generic error response (the sad elephant).
As a quick fix, I just replaced all ocurrences of %{title} with Hometown.

No ability to toot local-only?

I haven't found an option to set toots as local-only in the composer form. Maybe it got lost in one of the recent merges.

I forgot to run bundle exec rails assets:precompile, oops.

[mistydemeo]

Thank you! I'll try to get that HAML fixed. You wouldn't happen to remember which ones were broken, would you?

[nachtjasmin]

Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] ActionView::Template::Error (missing interpolation argument :title in "%{title} hosted on %{domain}" ({domain: "queer.group"} given))
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] Caused by: I18n::MissingInterpolationArgument (missing interpolation argument :title in "%{title} hosted on %{domain}" ({domain: "queer.group"} given))
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]
Jan 25 01:44:20 fedi-2 bundle[932007]: Information for: ActionView::Template::Error (missing interpolation argument :title in "%{title} hosted on %{domain}" ({domain: "queer.group"} given)):
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     3:
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     4: %meta{ name: 'description', content: description }/
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     5:
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     6: = opengraph 'og:site_name', t('about.hosted_on', domain: site_hostname)
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     7: = opengraph 'og:url', url_for(only_path: false)
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     8: = opengraph 'og:type', 'website'
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]     9: = opengraph 'og:title', instance_presenter.title
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3]
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] app/views/shared/_og.html.haml:6
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] app/views/home/index.html.haml:5
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] app/views/home/index.html.haml:1
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] app/controllers/concerns/localized.rb:11:in 'Localized#set_locale'
Jan 25 01:44:20 fedi-2 bundle[932007]: [864d4eb9-7e3b-42a5-8418-77f0fce547f3] lib/mastodon/middleware/socket_cleanup.rb:11:in 'Mastodon::Middleware::SocketCleanup#call'

So, should be one of the following two files:

• app/views/shared/_og.html.haml
• app/views/home/index.html.haml

[mistydemeo]

Thanks! Pushed a fix. I see that you figured out the local-only posting too.

[nachtjasmin]

@mistydemeo thank you! And yea, figured the local-only posting out as well, should've mentioned that explicitly again as well, my bad.

One of our users reported another minor issue: duplicate alt text badges.

(the post I took a screenshot of)

After applying this patch, they're gone.

diff --git a/app/javascript/mastodon/components/media_gallery.jsx b/app/javascript/mastodon/components/media_gallery.jsx
index a28ab26394..0920550cc6 100644
--- a/app/javascript/mastodon/components/media_gallery.jsx
+++ b/app/javascript/mastodon/components/media_gallery.jsx
@@ -113,10 +113,6 @@
       badges.push(<NoAltTextBadge key='no-alt' />);
     }
 
-    if (description?.length > 0) {
-      badges.push(<AltTextBadge key='alt' description={description} />);
-    }
-
     if (attachment.get('type') === 'unknown') {
       return (
         <div className={classNames('media-gallery__item', { standalone, 'media-gallery__item--tall': height === 100, 'media-gallery__item--wide': width === 100, 'media-missing-description': !hasMediaDescription  })} key={attachment.get('id')}>

Also, I can't express how grateful I am for the work you did with this PR, thank you! 🙏

[mistydemeo]

Good catch, thanks! Removed the duplicate alt text badge.

If you haven't seen any other issues, I think at this point the only two things remaining are:

• Finishing getting the CSS colours right in the two Hometown themes
• Ensuring that a user can't make a federating quote of a local-only post. I think we want two parts to this:
  1. Ensuring that, in the UI, the quote is forced to local-only if the quoted post is local-only.
  2. Ensuring that, on the backend, we reject an attempt to create a post in this state.

[mistydemeo]

Pushed a fix for the backend part of quote local-only permissions. Just need to handle the UI.

[mistydemeo]

Got the UI! Pushed a fix that disables the federation dropdown and sets it to "local-only" if you start a quote of a local-only post.

All we need now is the CSS, and we should be golden.

[nachtjasmin]

Tested the recent changes of the local-only handling. They work, to some extent, by which I mean they work with the web interface. I tested this behaviour with the iOS app Ivory as well and quote toots of local-only toots were rejected with client error 422. On the one hand, this is nice, shows that the validation is working.

On the other hand tho, Ivory doesn't know anything about local-only posts, which means there's nothing I can do within the app to get local-only quotes working. Not sure whether the API behaviour should be changed to silently change such posts to be local_only automatically, but it certainly would be a way to avoid this problem. ^^

[mistydemeo]

Yeah, I'm not sure what a better experience would be with Ivory not knowing about local-only status... I feel like forcing the status to local-only would be confusing for the user too, who'd have no idea why their post didn't federate since they couldn't see that the quoted post was local-only to begin with.

[mistydemeo]

Got some more CSS work in tonight, and mostly finished fairy-floss. Will finish up soon, then provide screenshots. Once I'm done I'll provide some screenshots so we can decide if anything else needs tweaking, then get on macaron.