Skip to content

Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
    • Help
    • Submit feedback
    • Contribute to GitLab
    • Switch to GitLab Next
  • Sign in / Register
Minds Backend - Engine
Minds Backend - Engine
  • Project
    • Project
    • Details
    • Activity
    • Releases
    • Cycle Analytics
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Charts
    • Locked Files
  • Issues 138
    • Issues 138
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Merge Requests 29
    • Merge Requests 29
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Charts
  • Registry
    • Registry
  • Packages
    • Packages
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Charts
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • Minds
  • Minds Backend - EngineMinds Backend - Engine
  • Merge Requests
  • !159

Open
Opened 3 weeks ago by Brian Hatchet@brianhatchet:speech_balloon:
  • Report abuse
Report abuse

WIP: Firehose api

New firehose admin endpoints for getting an unsorted list of activites from minds_badger and an admin endpoint that takes an activity guid and will mark it as moderated.

New managers and repositories for talking to elastic search Redis lock system with some super basic hash keying. These records shouldn't get very big, so we might want to KISS for now. Else, there are distributed locking libraries we should look into

It stores a key in moderation:entity_id:user_id format where the entity is the activity being moderated and user is the admin who "checked it out". These are set to a TTL of 15 minutes for now. When we query elastic search, we look for the opposite set (any locks that DON'T belong to the user) and for any activity that has not already been moderated. Requires adding two new fields to badger (in postman)

Edited 2 weeks ago by Brian Hatchet

Check out, review, and merge locally

Step 1. Fetch and check out the branch for this merge request

git fetch https://gitlab.com/brianhatchet/engine.git admin_firehose
git checkout -b brianhatchet/engine-admin_firehose FETCH_HEAD

Step 2. Review the changes locally

Step 3. Merge the branch and fix any conflicts that come up

git fetch origin
git checkout origin/master
git merge --no-ff brianhatchet/engine-admin_firehose

Step 4. Push the result of the merge to GitLab

git push origin master

Note that pushing to GitLab requires write access to this repository.

Tip: You can also checkout merge requests locally by following these guidelines.

Request to merge brianhatchet:admin_firehose into master
Open in Web IDE
  • Email patches
  • Plain diff
Pipeline #63781138 failed for 3cecc956 on brianhatchet:admin_firehose
      Requires 2 more approvals.
      This is a Work in Progress
      Allows commits from members who can merge to the target branch

      Deletes source branch

      You can merge this merge request manually using the
      • Discussion 14
      • Commits 20
      • Pipelines 19
      • Changes 19
      7/8 discussions resolved
      • Loading...
      • Brian Hatchet :speech_balloon: @brianhatchet changed milestone to %sprint: Dapper Dingo 3 weeks ago

        changed milestone to %sprint: Dapper Dingo

      • Brian Hatchet :speech_balloon: @brianhatchet added 7 commits 3 weeks ago

        added 7 commits

        • 269df1ee...1eaf8c06 - 5 commits from branch minds:epic/ReportingAndModeration
        • 50fabf98 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose
        • a1c2f3a6 - Fixing previous tests

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 3 weeks ago

        added 1 commit

        • e881f09a - Refactored to use the redis cache

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 3 weeks ago

        added 1 commit

        • 64942ca3 - Fixing unit tests by not overriding DI config in features. Thanks, Mark <3

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 37 commits 3 weeks ago

        added 37 commits

        • 64942ca3...5f29f688 - 36 commits from branch minds:epic/ReportingAndModeration
        • feb5a3b0 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 3 weeks ago
        Resolved by Mark Harding 6 hours ago
      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 3 weeks ago

        added 1 commit

        • efa38fcc - Updated tested to reflect the new elastic search query

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 3 commits 2 weeks ago

        added 3 commits

        • efa38fcc...9dbd95e5 - 2 commits from branch minds:epic/ReportingAndModeration
        • 8a3c9f0a - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

        added 1 commit

        • 96127343 - MR fixes. Performant ES query. Mapping fixes. Moved reconstitution to the manager

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

        added 1 commit

        • 66977958 - Moving save to the manager

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

        added 1 commit

        • 77c99287 - firehose manager reporting

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

        added 1 commit

        • b1701f83 - Bug fixes

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet unmarked as a Work In Progress 2 weeks ago

        unmarked as a Work In Progress

      • Brian Hatchet :speech_balloon: @brianhatchet changed title from WIP firehose api to Firehose api 2 weeks ago

        changed title from WIP firehose api to Firehose api

      • Brian Hatchet :speech_balloon: @brianhatchet changed the description 2 weeks ago

        changed the description

      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 2 weeks ago

        added 1 commit

        • c5cf7221 - Merging upstream

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 16 commits 2 weeks ago

        added 16 commits

        • c5cf7221...3927173f - 15 commits from branch minds:epic/ReportingAndModeration
        • 2dcea399 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 51 commits 1 week ago

        added 51 commits

        • 2dcea399...eadf4ebd - 50 commits from branch minds:epic/ReportingAndModeration
        • 74c34afb - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 2 commits 1 week ago

        added 2 commits

        • 9dc49442 - 1 commit from branch minds:epic/ReportingAndModeration
        • e3a25163 - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 6 commits 6 days ago

        added 6 commits

        • e3a25163...dd117ff0 - 5 commits from branch minds:epic/ReportingAndModeration
        • a3dcc56b - Merge remote-tracking branch 'upstream/epic/ReportingAndModeration' into admin_firehose

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet added 24 commits 2 days ago

        added 24 commits

        • bd6e7d55 - (feat) add report reasons to config endpoint
        • 25b05ce9 - (chore): ban spam domain
        • d07486fc - (chore): ban spammy email
        • b4a60db8 - (fix): Re-sync ES if expired boost was already completed (#448 (closed))
        • e9491d70 - Merge branch 'hotfix/es-boosts-resync' into 'master'
        • 56b60f52 - Merge branch 'sprint/ElevatedEagle.feat.report-reasons.added.minds-config.endpoint' into 'master'
        • cb33fe87 - Merge branch 'epic/ReportingAndModeration'
        • e0ced6e2 - (chore): content policy
        • a593916e - (chore): if channel reported for spam then do immediate ban
        • 1dc7c4b9 - [Sprint/FunnyFrog])(bug): Forcing NSFW reasons array into integers.
        • fcfd028f - Merge branch 'fix-nsfw-remind-inheritance-bug' into 'master'
        • 35a80e27 - (chore): change SEO to The Jury System
        • ef9255df - (fix): remove revoked from stats
        • 8634a913 - CORS enabled
        • c710bdef - Merge branch 'checkout-oauth' into 'master'
        • 8d0ccfc1 - Sprint/ElevatedEagle: Forced querystring params into lower case
        • 8bd9d70c - Merge branch 'force-lower-case-search-query-and-hashtags' into 'master'
        • 32b1f6ff - (chore): email on a friday
        • cf10fc88 - (chore); email on a friday...
        • e1ec640d - (fix): typos in email
        • 0c777244 - (chore): loop email iterator if no entities returned
        • 7546fd46 - (chore): dev code for promotions email
        • 827ae8e9 - (chore): email promotions to use markdown and include template key in hash
        • e605f46b - Merge remote-tracking branch 'upstream/master' into admin_firehose

        Compare with previous version

        Toggle commit list
      • Brian Hatchet :speech_balloon: @brianhatchet added 8 commits 8 hours ago

        added 8 commits

        • f75596ab - (chore): ban more domains
        • 01e1a432 - (chore): bring back channel level spam check for new channels
        • 85de3a3c - (chore): add another blocked domain
        • e40d62d3 - (chore): spam check comments
        • 23f0c499 - (chore): Appeals loop and awaiting TTL (#463)
        • 5b0ef158 - Merge branch 'hotfix/appeal-summons-tweaks-1' into 'master'
        • 294a8bc2 - (feat): support onchain boost label
        • f3d0b5ee - Merge remote-tracking branch 'upstream/master' into admin_firehose

        Compare with previous version

        Toggle commit list
      • Mark Harding @markeharding changed target branch from epic/ReportingAndModeration to master 6 hours ago

        changed target branch from epic/ReportingAndModeration to master

      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Resolved by Brian Hatchet 30 minutes ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on an old version of the diff 6 hours ago
        Resolved by Brian Hatchet 29 minutes ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Resolved by Brian Hatchet 29 minutes ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Resolved by Brian Hatchet 28 minutes ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Resolved by Mark Harding 5 hours ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Resolved by Brian Hatchet 27 minutes ago
      • Mark Harding
        Mark Harding @markeharding started a discussion on the diff 6 hours ago
        Entities/Activity.php
        691 697 return $this->ownerObj;
        692 698 }
        693 699
        700 public function setModeratorGuid($moderatorGuid) {
        • Mark Harding
          Mark Harding @markeharding · 6 hours ago
          Owner

          Can we get some phpdocs in ?

        Please register or sign in to reply
      • Brian Hatchet :speech_balloon: @brianhatchet added 1 commit 6 hours ago

        added 1 commit

        • 3cecc956 - WIP using top feed manager

        Compare with previous version

      • Brian Hatchet :speech_balloon: @brianhatchet marked as a Work In Progress from brianhatchet/engine@3cecc956 6 hours ago

        marked as a Work In Progress from brianhatchet/engine@3cecc956

      • You're only seeing other activity in the feed. To add a comment, switch to one of the following options.
      Please register or sign in to reply
      Assignee
      Brian Hatchet's avatar Brian Hatchet @brianhatchet
      Assign to
      sprint: Dapper Dingo
      Milestone
      sprint: Dapper Dingo
      Assign milestone
      None
      Time tracking
      No estimate or time spent
      0
      Labels
      None
      Assign labels
      • View project labels
      Lock merge request
      Unlocked
      2
      2 participants
      user avatar
      Brian Hatchet
      user avatar
      Mark Harding
      Reference: minds/engine!159