Skip to content

Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
    • Switch to GitLab Next
  • Sign in / Register
Minds Backend - Engine
Minds Backend - Engine
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
    • Locked Files
  • Issues 146
    • Issues 146
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 36
    • Merge Requests 36
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Registry
    • Registry
  • Packages
    • Packages
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • Minds
  • Minds Backend - EngineMinds Backend - Engine
  • Merge Requests
  • !159

Merged
Opened 1 month ago by Brian Hatchet@brianhatchet:speech_balloon:
  • Report abuse
Report abuse

Firehose api

New firehose endpoint uses the top feed algorithms to deliver reserved entities for users to moderate.

New managers and repositories for talking to elastic search via the top feed managers Redis lock system with some super fast hash keying. These records shouldn't get very big, so we might want to KISS for now. Else, there are distributed locking libraries we should look into

It stores a key in as set entity_id:user_id format where the entity is the activity being moderated and user is the admin who "checked it out". These are set to a TTL of 10 minutes for now. When we query elastic search, we look for the opposite set (any locks that DON'T belong to the user) and for any activity that has not already been moderated. Requires adding two new fields to badger (in postman)

Edited 1 week ago by Brian Hatchet
Request to merge brianhatchet:admin_firehose into master
  • Email patches
  • Plain diff
Pipeline #64494935 (#391) passed for 1a2d4ea5 on brianhatchet:admin_firehose
      Merge request approved. Approved by
      Marcelo Rivera
      Marcelo Rivera
      Mark Harding
      Mark Harding
      Rami Albatal
      Rami Albatal
      Emiliano Balbuena
      Emiliano Balbuena
      Ben Hayward
      Ben Hayward
      Martin Santangelo
      Martin Santangelo

      Merged by Mark Harding 8 hours ago

      The changes were merged into master with ba45f02b

      The source branch has been deleted

      Allows commits from members who can merge to the target branch

      Pipeline #65819255 (#751) passed for ba45f02b on master
            • Discussion 15
            • Commits 30
            • Pipelines 28
            • Changes 22
            7/8 discussions resolved
            • Loading...
            • Brian Hatchet :speech_balloon: @brianhatchet changed milestone to %sprint: Dapper Dingo 1 month ago

              changed milestone to %sprint: Dapper Dingo

            • Brian Hatchet :speech_balloon: @brianhatchet added 7 commits 1 month ago

              added 7 commits

              • 269df1ee...1eaf8c06 - 5 commits from branch minds:epic/ReportingAndModeration
              • 50fabf98 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose
              • a1c2f3a6 - Fixing previous tests

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 month ago

              added 1 commit

              • e881f09a - Refactored to use the redis cache

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 month ago

              added 1 commit

              • 64942ca3 - Fixing unit tests by not overriding DI config in features. Thanks, Mark <3

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 37 commits 1 month ago

              added 37 commits

              • 64942ca3...5f29f688 - 36 commits from branch minds:epic/ReportingAndModeration
              • feb5a3b0 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Mark Harding
              Mark Harding @markeharding started a discussion on the diff 1 month ago
              Resolved by Mark Harding 2 weeks ago
            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 month ago

              added 1 commit

              • efa38fcc - Updated tested to reflect the new elastic search query

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 3 commits 1 month ago

              added 3 commits

              • efa38fcc...9dbd95e5 - 2 commits from branch minds:epic/ReportingAndModeration
              • 8a3c9f0a - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 month ago

              added 1 commit

              • 96127343 - MR fixes. Performant ES query. Mapping fixes. Moved reconstitution to the manager

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 month ago

              added 1 commit

              • 66977958 - Moving save to the manager

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 4 weeks ago

              added 1 commit

              • 77c99287 - firehose manager reporting

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 4 weeks ago

              added 1 commit

              • b1701f83 - Bug fixes

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet unmarked as a Work In Progress 4 weeks ago

              unmarked as a Work In Progress

            • Brian Hatchet :speech_balloon: @brianhatchet changed title from WIP firehose api to Firehose api 4 weeks ago

              changed title from WIP firehose api to Firehose api

            • Brian Hatchet :speech_balloon: @brianhatchet changed the description 4 weeks ago

              changed the description

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 4 weeks ago

              added 1 commit

              • c5cf7221 - Merging upstream

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 16 commits 4 weeks ago

              added 16 commits

              • c5cf7221...3927173f - 15 commits from branch minds:epic/ReportingAndModeration
              • 2dcea399 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 51 commits 2 weeks ago

              added 51 commits

              • 2dcea399...eadf4ebd - 50 commits from branch minds:epic/ReportingAndModeration
              • 74c34afb - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 2 commits 2 weeks ago

              added 2 commits

              • 9dc49442 - 1 commit from branch minds:epic/ReportingAndModeration
              • e3a25163 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 6 commits 2 weeks ago

              added 6 commits

              • e3a25163...dd117ff0 - 5 commits from branch minds:epic/ReportingAndModeration
              • a3dcc56b - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 24 commits 2 weeks ago

              added 24 commits

              • bd6e7d55 - (feat) add report reasons to config endpoint
              • 25b05ce9 - (chore): ban spam domain
              • d07486fc - (chore): ban spammy email
              • b4a60db8 - (fix): Re-sync ES if expired boost was already completed (#448 (closed))
              • e9491d70 - Merge branch 'hotfix/es-boosts-resync' into 'master'
              • 56b60f52 - Merge branch 'sprint/ElevatedEagle.feat.report-reasons.added.minds-config.endpoint' into 'master'
              • cb33fe87 - Merge branch 'epic/ReportingAndModeration'
              • e0ced6e2 - (chore): content policy
              • a593916e - (chore): if channel reported for spam then do immediate ban
              • 1dc7c4b9 - [Sprint/FunnyFrog])(bug): Forcing NSFW reasons array into integers.
              • fcfd028f - Merge branch 'fix-nsfw-remind-inheritance-bug' into 'master'
              • 35a80e27 - (chore): change SEO to The Jury System
              • ef9255df - (fix): remove revoked from stats
              • 8634a913 - CORS enabled
              • c710bdef - Merge branch 'checkout-oauth' into 'master'
              • 8d0ccfc1 - Sprint/ElevatedEagle: Forced querystring params into lower case
              • 8bd9d70c - Merge branch 'force-lower-case-search-query-and-hashtags' into 'master'
              • 32b1f6ff - (chore): email on a friday
              • cf10fc88 - (chore); email on a friday...
              • e1ec640d - (fix): typos in email
              • 0c777244 - (chore): loop email iterator if no entities returned
              • 7546fd46 - (chore): dev code for promotions email
              • 827ae8e9 - (chore): email promotions to use markdown and include template key in hash
              • e605f46b - Merge remote-tracking branch 'upstream/master' into admin_firehose

              Compare with previous version

              Toggle commit list
            • Brian Hatchet :speech_balloon: @brianhatchet added 8 commits 2 weeks ago

              added 8 commits

              • f75596ab - (chore): ban more domains
              • 01e1a432 - (chore): bring back channel level spam check for new channels
              • 85de3a3c - (chore): add another blocked domain
              • e40d62d3 - (chore): spam check comments
              • 23f0c499 - (chore): Appeals loop and awaiting TTL (#463 (closed))
              • 5b0ef158 - Merge branch 'hotfix/appeal-summons-tweaks-1' into 'master'
              • 294a8bc2 - (feat): support onchain boost label
              • f3d0b5ee - Merge remote-tracking branch 'upstream/master' into admin_firehose

              Compare with previous version

              Toggle commit list
            • Mark Harding @markeharding changed target branch from epic/ReportingAndModeration to master 2 weeks ago

              changed target branch from epic/ReportingAndModeration to master

            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Brian Hatchet 1 week ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Brian Hatchet 1 week ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Brian Hatchet 1 week ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Brian Hatchet 1 week ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Mark Harding 2 weeks ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Resolved by Brian Hatchet 1 week ago
            • Mark Harding
              Mark Harding @markeharding started a discussion on an old version of the diff 2 weeks ago
              Last updated by Brian Hatchet 1 week ago
              Entities/Activity.php
              691 697 return $this->ownerObj;
              692 698 }
              693 699
              700 public function setModeratorGuid($moderatorGuid) {
              • Mark Harding
                Mark Harding @markeharding · 2 weeks ago
                Owner

                Can we get some phpdocs in ?

              • Brian Hatchet :speech_balloon: @brianhatchet changed this line in version 21 of the diff 1 week ago

                changed this line in version 21 of the diff

              Please register or sign in to reply
            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

              added 1 commit

              • 3cecc956 - WIP using top feed manager

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet marked as a Work In Progress from brianhatchet/engine@3cecc956 2 weeks ago

              marked as a Work In Progress from brianhatchet/engine@3cecc956

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 week ago

              added 1 commit

              • cc861b6b - MR changes and everything working edition

              Compare with previous version

            • Mark Harding @markeharding added 1 commit 1 week ago

              added 1 commit

              • 0962e34f - (chore): remove un-needed changes

              Compare with previous version

            • Mark Harding @markeharding added 1 commit 1 week ago

              added 1 commit

              • 9ad3dc0d - (chore): remove un-needed changes

              Compare with previous version

            • Mark Harding @markeharding added 1 commit 1 week ago

              added 1 commit

              • 84be2d49 - (chore): remove cassandra log

              Compare with previous version

            • Mark Harding @markeharding added 1 commit 1 week ago

              added 1 commit

              • eabf1086 - (fix): avoid full db scan on pattern

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 5 commits 1 week ago

              added 5 commits

              • eabf1086...9b4c685a - 2 commits from branch minds:master
              • 3a70ba3f - Merge branch 'master' of gitlab.com:minds/engine into admin_firehose
              • b5505f87 - Merge remote-tracking branch 'origin/admin_firehose' into admin_firehose
              • 3d8cdeff - Everything working and complete.

              Compare with previous version

              Toggle commit list
            • Brian Hatchet :speech_balloon: @brianhatchet unmarked as a Work In Progress 1 week ago

              unmarked as a Work In Progress

            • Brian Hatchet :speech_balloon: @brianhatchet changed the description 1 week ago

              changed the description

            • Brian Hatchet :speech_balloon: @brianhatchet changed milestone to %sprint: Funny Frog 1 week ago

              changed milestone to %sprint: Funny Frog

            • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 1 week ago

              added 1 commit

              • a878510b - Actually filing decisions with the jury manager

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 9 commits 1 week ago

              added 9 commits

              • a878510b...678abba9 - 8 commits from branch minds:master
              • 43c9b71f - Merge remote-tracking branch 'upstream/master' into admin_firehose

              Compare with previous version

            • Brian Hatchet :speech_balloon: @brianhatchet added 8 commits 1 week ago

              added 8 commits

              • 43c9b71f...75d495b6 - 7 commits from branch minds:master
              • 1a2d4ea5 - Merge remote-tracking branch 'upstream/master' into admin_firehose

              Compare with previous version

            • Mark Harding @markeharding approved this merge request 1 day ago

              approved this merge request

            • Mark Harding
              Mark Harding @markeharding · 1 day ago
              Owner

              @all please approve

            • Marcelo Rivera @eiennohi approved this merge request 19 hours ago

              approved this merge request

            • Mark Harding @markeharding merged 8 hours ago

              merged

            • Mark Harding @markeharding mentioned in commit ba45f02b 8 hours ago

              mentioned in commit ba45f02b

            • You're only seeing other activity in the feed. To add a comment, switch to one of the following options.
            Please register or sign in to reply
            Assignee
            Brian Hatchet's avatar Brian Hatchet @brianhatchet
            Assign to
            sprint: Funny Frog
            Milestone
            sprint: Funny Frog
            Assign milestone
            None
            Time tracking
            No estimate or time spent
            0
            Labels
            None
            Assign labels
            • View project labels
            Lock merge request
            Unlocked
            15
            15 participants
            user avatar
            Rami Albatal
            user avatar
            Emiliano Balbuena
            user avatar
            Ben Hayward
            user avatar
            Martin Santangelo
            user avatar
            Brian Hatchet
            user avatar
            Mark Harding
            user avatar
            Marcelo Rivera
            Reference: minds/engine!159

            Revert this merge request

            This will create a new commit in order to revert the existing changes.

            Switch branch
            Cancel
            A new branch will be created in your fork and a new merge request will be started.

            Cherry-pick this merge request

            Switch branch
            Cancel
            A new branch will be created in your fork and a new merge request will be started.