does this increment comment count to 75 from 74 with trigger?

a 2nd comment today

comment activity 1

logs of the comment delete so far:

federation.2023-07-14:2023-07-14T17:52:44.073751Z DEBUG federation: apub.activities.delete.verify Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/comment/1266537", query: None, fragment: None }

federation.2023-07-14:2023-07-14T17:52:44.076617Z DEBUG federation: apub.lib.rs.insert_activity DbUrl(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/activities/delete/5f60929a-72ad-47c6-a75c-1f258ad7556c", query: None, fragment: None }) Object {"actor": String("https://lemmy.world/u/BitOneZero"), "to": Array [String("https://www.w3.org/ns/activitystreams#Public")], "object": String("https://lemmy.world/comment/1266537"), "type": String("Delete"), "id": String("https://lemmy.world/activities/delete/5f60929a-72ad-47c6-a75c-1f258ad7556c"), "audience": String("https://bulletintree.com/c/lemmy_testing0000"), "cc": Array [String("https://bulletintree.com/c/lemmy_testing0000")]}

logs of the comment restore (undelete) so far:

federation.2023-07-14:2023-07-14T17:54:29.938688Z DEBUG federation: apub.activities.delete.verify Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/comment/1266537", query: None, fragment: None }

federation.2023-07-14:2023-07-14T17:54:29.942701Z DEBUG federation: apub.activities.delete.verify Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/comment/1266537", query: None, fragment: None }

federation.2023-07-14:2023-07-14T17:54:29.945402Z DEBUG federation: apub.lib.rs.insert_activity DbUrl(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/activities/undo/2b03187b-8f46-4d12-b7d4-daa67b8e33c5", query: None, fragment: None }) Object {"actor": String("https://lemmy.world/u/BitOneZero"), "to": Array [String("https://www.w3.org/ns/activitystreams#Public")], "object": Object {"actor": String("https://lemmy.world/u/BitOneZero"), "to": Array [String("https://www.w3.org/ns/activitystreams#Public")], "object": String("https://lemmy.world/comment/1266537"), "type": String("Delete"), "id": String("https://lemmy.world/activities/delete/82570b01-605e-4b75-9276-eb30b7ad964d"), "audience": String("https://bulletintree.com/c/lemmy_testing0000"), "cc": Array [String("https://bulletintree.com/c/lemmy_testing0000")]}, "type": String("Undo"), "id": String("https://lemmy.world/activities/undo/2b03187b-8f46-4d12-b7d4-daa67b8e33c5"), "audience": String("https://bulletintree.com/c/lemmy_testing0000"), "cc": Array [String("https://bulletintree.com/c/lemmy_testing0000")]}

Community home database record id: https://bulletintree.com/comment/761193

Database record ID for comment on community home server: https://bulletintree.com/comment/759545

Adding even more logging now

I'm adding additional logging on home server

Takes about several minutes to recompile lemmy_server after making additions to the code, so waiting on that

Ok, the comment now shows "deleted by creator" on the community home server.

Shows up fine on home server

ok, I have heavy logging turned on now for my server bulletintree.com - so time to start testing comment delete activity.

Still not deleted here

Test comment 2023-07-13T16:17:29Z

Test comment shortly after 2023-07-13T16:02:33Z

test comment

comment on delete

test comment on test post

Commenting on this test post before I delete it

SQL: UPDATE "post" SET "featured_community" = $1 WHERE ("post"."id" = $2) RETURNING "post"."id", "post"."name", "post"."url", "post"."body", "post"."creator_id", "post"."community_id", "post"."removed", "post"."locked", "post"."published", "post"."updated", "post"."deleted", "post"."nsfw", "post"."embed_title", "post"."embed_description", "post"."thumbnail_url", "post"."ap_id", "post"."local", "post"."embed_video_url", "post"."language_id", "post"."featured_community", "post"."featured_local"

ok, I am going to delete my own post

It's right there, June 22, I was begging https://lemm.ee/comment/350801 - we had more than a week before the Reddit API shutdown - but none of the big servers ran pg_stat_statements and shared the data. I got so frustrated I created this dumb community because I felt like the whole project was ignoring pg_stat_statements and finding out what was going on with the high-activity servers and why they were constantly crashing. Sanity with the project has returned!

You have to be really careful with ORM code that looks simple (Lemmy uses Diesel)... it fetches full fields on every table, it will bring in a million rows when you aren't careful.

ORIGINATE HERE. Spent some hours tracing the code, inserting logging that I watched based on a UPVOTE being federated.

diff --git a/crates/api/src/comment/like.rs b/crates/api/src/comment/like.rs
index 6c4bdebc7..063249a7e 100644
--- a/crates/api/src/comment/like.rs
+++ b/crates/api/src/comment/like.rs
@@ -17,6 +17,7 @@ use lemmy_db_schema::{
 };
 use lemmy_db_views::structs::{CommentView, LocalUserView};
 use lemmy_utils::error::LemmyError;
+use tracing::warn;
 
 #[async_trait::async_trait(?Send)]
 impl Perform for CreateCommentLike {
@@ -25,6 +26,7 @@ impl Perform for CreateCommentLike {
   #[tracing::instrument(skip(context))]
   async fn perform(&self, context: &Data<LemmyContext>) -> Result<CommentResponse, LemmyError> {
     let data: &CreateCommentLike = self;
+    // Here we are with that database read on every single like for site data.
     let local_site = LocalSite::read(context.pool()).await?;
     let local_user_view = local_user_view_from_jwt(&data.auth, context).await?;
 
@@ -62,16 +64,20 @@ impl Perform for CreateCommentLike {
     // Remove any likes first
     let person_id = local_user_view.person.id;
 
+    // does database transaction, despite next one doing update condition?
     CommentLike::remove(context.pool(), person_id, comment_id).await?;
 
     // Only add the like if the score isnt 0
     let do_add = like_form.score != 0 && (like_form.score == 1 || like_form.score == -1);
     if do_add {
+      // this does database transaction. it await for result
       CommentLike::like(context.pool(), &like_form)
         .await
         .map_err(|e| LemmyError::from_error_message(e, "couldnt_like_comment"))?;
     }
 
+    warn!("zebratrace100 vote do_add {:?}", do_add);
+
     build_comment_response(
       context,
       comment_id,
diff --git a/crates/apub/src/activities/community/announce.rs b/crates/apub/src/activities/community/announce.rs
index 116b02726..62af17a1a 100644
--- a/crates/apub/src/activities/community/announce.rs
+++ b/crates/apub/src/activities/community/announce.rs
@@ -24,6 +24,7 @@ use lemmy_api_common::context::LemmyContext;
 use lemmy_utils::error::LemmyError;
 use serde_json::Value;
 use url::Url;
+use tracing::warn;
 
 #[async_trait::async_trait]
 impl ActivityHandler for RawAnnouncableActivities {
@@ -93,6 +94,11 @@ impl AnnounceActivity {
   ) -> Result<(), LemmyError> {
     let announce = AnnounceActivity::new(object.clone(), community, context)?;
     let inboxes = community.get_follower_inboxes(context).await?;
+
+    // example, local community with remote subscribers generates this: zebratrace500 like activity inboxes [Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.ml")), port: None, path: "/inbox", query: None, fragment: None }, Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.management")), port: None, path: "/inbox", query: None, fragment: None }, Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.austinite.online")), port: None, path: "/inbox", query: None, fragment: None }, Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmywinks.xyz")), port: None, path: "/inbox", query: None, fragment: None }, Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.asc6.org")), port: None, path: "/inbox", query: None, fragment: None }]
+    // example, local community with no remote subscribers, generates this: zebratrace500 like activity inboxes []
+    warn!("zebratrace500 like activity inboxes {:?}", inboxes);
+
     send_lemmy_activity(context, announce, community, inboxes.clone(), false).await?;
 
     // Pleroma and Mastodon can't handle activities like Announce/Create/Page. So for
diff --git a/crates/apub/src/activities/community/mod.rs b/crates/apub/src/activities/community/mod.rs
index 010bad4f4..58ae1b732 100644
--- a/crates/apub/src/activities/community/mod.rs
+++ b/crates/apub/src/activities/community/mod.rs
@@ -9,6 +9,7 @@ use lemmy_api_common::context::LemmyContext;
 use lemmy_db_schema::source::person::PersonFollower;
 use lemmy_utils::error::LemmyError;
 use url::Url;
+use tracing::warn;
 
 pub mod announce;
 pub mod collection_add;
@@ -51,6 +52,8 @@ pub(crate) async fn send_activity_in_community(
     );
   }
 
+  warn!("zebratrace300 send_activity_in_community vote local? {:?}", community.local);
+
   if community.local {
     // send directly to community followers
     AnnounceActivity::send(activity.clone().try_into()?, community, context).await?;
@@ -59,6 +62,14 @@ pub(crate) async fn send_activity_in_community(
     inboxes.push(community.shared_inbox_or_inbox());
   }
 
+  // example output, remote homed community: zebratrace301 send_activity_in_community vote inboxes [Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("beehaw.org")), port: None, path: "/inbox", query: None, fragment: None }]
+  // example output, local homed community with no remote followers: zebratrace301 send_activity_in_community vote inboxes []
+  // same output even when local community has a remote follower: zebratrace301 send_activity_in_community vote inboxes []
+  warn!("zebratrace301 send_activity_in_community vote inboxes {:?}", inboxes);
+
   send_lemmy_activity(context, activity.clone(), actor, inboxes, false).await?;
+
+  warn!("zebratrace302 send_activity_in_community after");
+
   Ok(())
 }
diff --git a/crates/apub/src/activities/voting/mod.rs b/crates/apub/src/activities/voting/mod.rs
index 8bae05577..17d89557f 100644
--- a/crates/apub/src/activities/voting/mod.rs
+++ b/crates/apub/src/activities/voting/mod.rs
@@ -31,6 +31,7 @@ use lemmy_utils::error::LemmyError;
 
 pub mod undo_vote;
 pub mod vote;
+use tracing::warn;
 
 #[async_trait::async_trait]
 impl SendActivity for CreatePostLike {
@@ -76,6 +77,7 @@ impl SendActivity for CreateCommentLike {
   }
 }
 
+// this send_activity is for votes, look at the score parameter
 async fn send_activity(
   object_id: ObjectId<PostOrComment>,
   community_id: CommunityId,
@@ -89,6 +91,10 @@ async fn send_activity(
     .await?
     .into();
 
+  // production testing reveals that this code is hit with a user doing a vote on local or federated comment.
+  // production testing reveals that when a local user votes, even in a community with zero remote subscribers, this has a hefty &actor data payload. which is just the user who did the vote... lots of overhead to vote on a comment?
+  warn!("zebratrace200 send_activity vote {:?}", &actor);
+
   // score of 1 means upvote, -1 downvote, 0 undo a previous vote
   if score != 0 {
     let vote = Vote::new(object_id, &actor, &community, score.try_into()?, context)?;
diff --git a/crates/apub/src/activities/voting/undo_vote.rs b/crates/apub/src/activities/voting/undo_vote.rs
index bcb8ee406..44da390f3 100644
--- a/crates/apub/src/activities/voting/undo_vote.rs
+++ b/crates/apub/src/activities/voting/undo_vote.rs
@@ -12,6 +12,7 @@ use crate::{
   },
   PostOrComment,
 };
+// Ok, federation is involved here in this class somehow
 use activitypub_federation::{
   config::Data,
   kinds::activity::UndoType,
diff --git a/crates/apub/src/activities/voting/vote.rs b/crates/apub/src/activities/voting/vote.rs
index 7f36ed471..ee72563ed 100644
--- a/crates/apub/src/activities/voting/vote.rs
+++ b/crates/apub/src/activities/voting/vote.rs
@@ -12,6 +12,7 @@ use crate::{
   },
   PostOrComment,
 };
+// Ok, federation is involved somehow here
 use activitypub_federation::{
   config::Data,
   fetch::object_id::ObjectId,
@@ -22,6 +23,7 @@ use lemmy_api_common::context::LemmyContext;
 use lemmy_db_schema::source::local_site::LocalSite;
 use lemmy_utils::error::LemmyError;
 use url::Url;
+use tracing::warn;
 
 impl Vote {
   pub(in crate::activities::voting) fn new(
@@ -57,6 +59,7 @@ impl ActivityHandler for Vote {
   #[tracing::instrument(skip_all)]
   async fn verify(&self, context: &Data<LemmyContext>) -> Result<(), LemmyError> {
     let community = self.community(context).await?;
+    warn!("zebratrace000 vote verify {:?}", &self.actor);
     verify_person_in_community(&self.actor, &community, context).await?;
     let enable_downvotes = LocalSite::read(context.pool())
       .await
@@ -70,6 +73,8 @@ impl ActivityHandler for Vote {
 
   #[tracing::instrument(skip_all)]
   async fn receive(self, context: &Data<LemmyContext>) -> Result<(), LemmyError> {
+    // based on testing with a client locally, this code path is only used for federatio incoming votes?
+    warn!("zebratrace000 vote receive {:?}", &self.id);
     insert_activity(&self.id, &self, false, true, context).await?;
     let actor = self.actor.dereference(context).await?;
     let object = self.object.dereference(context).await?;
diff --git a/crates/apub/src/objects/community.rs b/crates/apub/src/objects/community.rs
index 17476e9f8..e437c61ad 100644
--- a/crates/apub/src/objects/community.rs
+++ b/crates/apub/src/objects/community.rs
@@ -33,6 +33,7 @@ use lemmy_utils::{
 };
 use std::ops::Deref;
 use tracing::debug;
+use tracing::warn;
 use url::Url;
 
 #[derive(Clone, Debug)]
@@ -207,6 +208,8 @@ impl ApubCommunity {
       })
       .collect();
 
+    warn!("zebratrace600 is every single federated outbound Like hitting this poiint?");
+
     Ok(inboxes)
   }
 }

ok, so inserting something into the Rust code at the API front door

The rate-limit logic is something to trace in the code and understand, as I assume that blocks before getting into SQL?

  /// Returns true if the request passed the rate limit, false if it failed and should be rejected.
  pub fn check(self, ip_addr: IpAddr) -> bool {
    // Does not need to be blocking because the RwLock in settings never held across await points,
    // and the operation here locks only long enough to clone
    let mut guard = self
      .rate_limit
      .lock()
      .expect("Failed to lock rate limit mutex for reading");
    let rate_limit = &guard.rate_limit_config;

    let (kind, interval) = match self.type_ {
      RateLimitType::Message => (rate_limit.message, rate_limit.message_per_second),
      RateLimitType::Post => (rate_limit.post, rate_limit.post_per_second),
      RateLimitType::Register => (rate_limit.register, rate_limit.register_per_second),
      RateLimitType::Image => (rate_limit.image, rate_limit.image_per_second),
      RateLimitType::Comment => (rate_limit.comment, rate_limit.comment_per_second),
      RateLimitType::Search => (rate_limit.search, rate_limit.search_per_second),
    };
    let limiter = &mut guard.rate_limiter;

    limiter.check_rate_limit_full(self.type_, ip_addr, kind, interval, InstantSecs::now())
  }

My server is getting federation inbound messages. I have just myself as local user. But federation protocols have a lot of outbound HTTP fetches:

Jul 01 19:04:05 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:05.496130Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=34622a86-e6ef-4a62-8514-f191a5289465}: activitypub_federation::fetch: Fetching remote object https://kbin.social/u/beesyrup
Jul 01 19:04:05 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:05.949767Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=ff4a97a8-a93b-4b37-84d9-e5ab01c3ee67}:receive:verify: activitypub_federation::fetch: Fetching remote object https://lemmy.world/comment/697555
Jul 01 19:04:06 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:06.029375Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=34622a86-e6ef-4a62-8514-f191a5289465}:from_json:from_json: activitypub_federation::fetch: Fetching remote object https://kbin.social/
Jul 01 19:04:06 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:06.368602Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=34622a86-e6ef-4a62-8514-f191a5289465}:receive: activitypub_federation::fetch: Fetching remote object https://kbin.social/m/RedditMigration/t/122349
Jul 01 19:04:06 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:06.699349Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=162a4d97-2951-4000-85c0-35d252c22faa}:receive:verify:verify_person_in_community: activitypub_federation::fetch: Fetching remote object https://lemmy.world/u/asuratva
Jul 01 19:04:08 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:08.074109Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=eed763b3-83af-4d5a-8190-8563a3ebf9ed}:receive:verify:verify_person_in_community: activitypub_federation::fetch: Fetching remote object https://infosec.pub/u/techviator
Jul 01 19:04:11 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:11.738901Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=e19f40f4-4840-457d-a8cb-3d5a37fa355c}:receive:verify:verify_person_in_community: activitypub_federation::fetch: Fetching remote object https://feddit.de/u/Quetzacoatl
Jul 01 19:04:13 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:13.662622Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=0b674845-0c58-40e8-9da8-f0b7b90a442f}:receive: activitypub_federation::fetch: Fetching remote object https://kbin.social/m/RedditMigration/t/122349
Jul 01 19:04:14 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:14.068155Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=b7336fc3-eebb-423d-aa8e-6636c429af84}:receive:verify: activitypub_federation::fetch: Fetching remote object https://lemmy.world/comment/694930
Jul 01 19:04:15 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T19:04:15.996734Z  INFO HTTP request{http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=3506dcc3-f9ef-4620-ab12-1261668bcfb2}:receive:verify: activitypub_federation::fetch: Fetching remote object https://lemmy.world/comment/694973

Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T18:35:13.897361Z  WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: Other errors which are not explicitly handled
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]: Caused by:
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:     0: error sending request for url (https://lemmy.world/comment/672548): operation timed out
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:     1: operation timed out
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:    0: lemmy_apub::activities::voting::vote::verify
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:              at crates/apub/src/activities/voting/vote.rs:57
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:    1: lemmy_apub::activities::community::announce::receive
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:              at crates/apub/src/activities/community/announce.rs:141
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:    2: lemmy_server::root_span_builder::HTTP request
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:            with http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=a8f3afdf-f1d7-4ca4-889c-2b564740caa8 http.status_code=400 otel.status_code="OK"
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:              at src/root_span_builder.rs:16
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]: LemmyError { message: None, inner: Other errors which are not explicitly handled
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]: Caused by:
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:     0: error sending request for url (https://lemmy.world/comment/672548): operation timed out
Jul 01 18:35:13 instance-20230117-wakereality0 lemmy_server[2358887]:     1: operation timed out, context: SpanTrace [{ target: "lemmy_apub::activities::voting::vote", name: "verify", file: "crates/apub/src/activities/voting/vote.rs", line: 57 }, { target: "lemmy_apub::activities::community::announce", name: "receive", file: "crates/apub/src/activities/community/announce.rs", line: 141 }, { target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "\u{1b}[3mhttp.method\u{1b}[0m\u{1b}[2m=\u{1b}[0mPOST \u{1b}[3mhttp.scheme\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"http\" \u{1b}[3mhttp.host\u{1b}[0m\u{1b}[2m=\u{1b}[0mbulletintree.com \u{1b}[3mhttp.target\u{1b}[0m\u{1b}[2m=\u{1b}[0m/inbox \u{1b}[3motel.kind\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"server\" \u{1b}[3mrequest_id\u{1b}[0m\u{1b}[2m=\u{1b}[0ma8f3afdf-f1d7-4ca4-889c-2b564740caa8 \u{1b}[3mhttp.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m400 \u{1b}[3motel.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"OK\"", file: "src/root_span_builder.rs", line: 16 }] }

if this is the 10-second HTTP timeout I've raised issue on, that's a lot of error logging for what could be a single line of log about timeout.

server logs

Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T18:30:00.365330Z  WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: duplicate key value violates unique constraint "idx_activity_ap_id"
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:    0: lemmy_apub::insert_activity
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:            with ap_id=Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("lemmy.world")), port: None, path: "/activities/announce/1d719e46-9faa-4121-9cf0-9373d83b09a4", query: None, fragment: None } local=false sensitive=false
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:              at crates/apub/src/lib.rs:159
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:    1: lemmy_apub::activities::community::announce::receive
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:              at crates/apub/src/activities/community/announce.rs:141
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:    2: lemmy_server::root_span_builder::HTTP request
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:            with http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=5e5de3a2-e76c-460a-9a75-4a84038836e5 http.status_code=400 otel.status_code="OK"
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]:              at src/root_span_builder.rs:16
Jul 01 18:30:00 instance-20230117-wakereality0 lemmy_server[2358887]: LemmyError { message: None, inner: duplicate key value violates unique constraint "idx_activity_ap_id", context: SpanTrace [{ target: "lemmy_apub", name: "insert_activity", fields: "\u{1b}[3map_id\u{1b}[0m\u{1b}[2m=\u{1b}[0mUrl { scheme: \"https\", cannot_be_a_base: false, username: \"\", password: None, host: Some(Domain(\"lemmy.world\")), port: None, path: \"/activities/announce/1d719e46-9faa-4121-9cf0-9373d83b09a4\", query: None, fragment: None } \u{1b}[3mlocal\u{1b}[0m\u{1b}[2m=\u{1b}[0mfalse \u{1b}[3msensitive\u{1b}[0m\u{1b}[2m=\u{1b}[0mfalse", file: "crates/apub/src/lib.rs", line: 159 }, { target: "lemmy_apub::activities::community::announce", name: "receive", file: "crates/apub/src/activities/community/announce.rs", line: 141 }, { target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "\u{1b}[3mhttp.method\u{1b}[0m\u{1b}[2m=\u{1b}[0mPOST \u{1b}[3mhttp.scheme\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"http\" \u{1b}[3mhttp.host\u{1b}[0m\u{1b}[2m=\u{1b}[0mbulletintree.com \u{1b}[3mhttp.target\u{1b}[0m\u{1b}[2m=\u{1b}[0m/inbox \u{1b}[3motel.kind\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"server\" \u{1b}[3mrequest_id\u{1b}[0m\u{1b}[2m=\u{1b}[0m5e5de3a2-e76c-460a-9a75-4a84038836e5 \u{1b}[3mhttp.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m400 \u{1b}[3motel.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"OK\"", file: "src/root_span_builder.rs", line: 16 }] }

server logs running 0.18.1rc4

Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]: 2023-07-01T18:26:00.678366Z  WARN Error encountered while processing the incoming HTTP request: lemmy_server::root_span_builder: data did not match any variant of untagged enum AnnouncableActivities
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]:    0: lemmy_apub::activities::community::announce::receive
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]:              at crates/apub/src/activities/community/announce.rs:46
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]:    1: lemmy_server::root_span_builder::HTTP request
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]:            with http.method=POST http.scheme="http" http.host=bulletintree.com http.target=/inbox otel.kind="server" request_id=1b28cfff-b0d5-437c-a296-0d7b2cf3bc78 http.status_code=400 otel.status_code="OK"
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]:              at src/root_span_builder.rs:16
Jul 01 18:26:00 instance-20230117-wakereality0 lemmy_server[2358887]: LemmyError { message: None, inner: data did not match any variant of untagged enum AnnouncableActivities, context: SpanTrace [{ target: "lemmy_apub::activities::community::announce", name: "receive", file: "crates/apub/src/activities/community/announce.rs", line: 46 }, { target: "lemmy_server::root_span_builder", name: "HTTP request", fields: "\u{1b}[3mhttp.method\u{1b}[0m\u{1b}[2m=\u{1b}[0mPOST \u{1b}[3mhttp.scheme\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"http\" \u{1b}[3mhttp.host\u{1b}[0m\u{1b}[2m=\u{1b}[0mbulletintree.com \u{1b}[3mhttp.target\u{1b}[0m\u{1b}[2m=\u{1b}[0m/inbox \u{1b}[3motel.kind\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"server\" \u{1b}[3mrequest_id\u{1b}[0m\u{1b}[2m=\u{1b}[0m1b28cfff-b0d5-437c-a296-0d7b2cf3bc78 \u{1b}[3mhttp.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m400 \u{1b}[3motel.status_code\u{1b}[0m\u{1b}[2m=\u{1b}[0m\"OK\"", file: "src/root_span_builder.rs", line: 16 }] }