New Management for Pushshift
Greetings Pushshift community!
This message is to inform you that Pushshift’s management has officially been transferred to the non-profit NCRI (Network Contagion Research Institute - [www.networkcontagion.us](http://www.networkcontagion.us/))
Like all of you, we have found Pushshift to be enormously valuable in providing data that helps us understand the impact of social media on the world around us. We’ve also recognized that Pushshift has not had the necessary staff support to be responsive to technical questions and inquiries.
We’d like to remind you that Push shift has been relying on donations since its inception to provide its services to the community. Now that NCRI has assumed management of Pushshift, we will strive to professionalize our service levels and response times to any of your questions or concerns. Please donate to NCRI help us maintain and develop Push shift. [https://www.paypal.com/US/fundraiser/charity/3521050](https://www.paypal.com/US/fundraiser/charity/3521050). We look forward to becoming more engaged with the Pushshift community and are thankful for the incredible contributions so many of you are making to the research community and beyond.
Feel free to ping us on this Reddit account directly with questions, or email us at pushshift-support@ncri.io and we look forward to hearing from you.
Could you please send the mods a modmail with proof that this account is legitimate?
Thanks.
Hi u/safrax, hope my MOD status is sufficient to ensure legitimacy.
Hey everyone -- Jason here. I want to clear the air and help explain some of the changes that have been happening lately. When I started Pushshift in 2015/2016, it was a very small service used by a handful of programmers and also by researchers who wanted massive amounts of Reddit data for research purposes. Since that time, it has grown into a service that gets over 1.13 billion hits per month by over one million unique visitors.
As time went on, I was simply overwhelmed with support requests, adding additional features and just keeping things running smoothly. Literally it was all I worked on for 14+ hours a day and over weekends. I did this while also becoming a primary caregiver for an immediately family member dealing with a major health issue.
I started working with the NCRI non-profit group three years ago and they provided a lot of support behind the scenes. I felt it was a good marriage to keep the community thriving and expanding, so we made more formal agreements to work together and partner with one another.
The Pushshift-Support user is operated by a trusted member of the NCRI group and will help provide support and further communication efforts for the expanding community. It also gives me an opportunity to focus on improving Pushshift and advancing the original cause that I always stood 100% behind -- to give the research community better access to social media data to help keep social media communities engagement more transparent for researchers to better understand since disinformation is a constantly growing problem for society.
I am happy to answer questions but this is really me Jason. I'm happy to take a call with one of the moderators to prove my identity or to confirm via Twitter, etc. -- I have not been hacked.
Pushshift will continue to provide free access to researchers. Money provided via Patreon will continue to be used to further the development of Pushshift. However, if donations are made via Paypal to NCRI, NCRI is a registered 501-c3 non-profit which can be used for taxation purposes if donations are made via the NCRI paypal account. Money made through that account will be used to improve and support Pushshift services.
Again, I apologize for the lateness in responding but the past couple months have been overwhelming on a personal level as we have moved to a COLO, hired additional engineers and have worked to continue to improve the health and robustness of Pushshift services while I have had to deal with personal caregiver issues. I want to thank the community and I'll check back again shortly to answer any questions.
Jason
Thanks for posting Jason, and thanks for all your work over the years.
Do you know if the NCRI team is planning to make any substantial changes to how pushshift runs? From how removals are processed, to whether they will implement API tokens and charge for higher levels of access. There's also the long list of bugs in the top comment here that need addressing.
I live in the DC area and would be happy to buy you a beer if you ever had the time.
Squints That's exactly what a SITM impersonator would say...
To be fair, I was far more suspicious when m.vea turned into Reddit's biggest
NFTCollectible Avatar aficionado.How big is the team now? Perhaps it would be beneficial to host an AMA here with the team to answer the community's questions.
Do you intend to monetize Pushshift (i.e. charge for access)?
When did this transfer actually occur? Pushshift has been presented as part of the NCRI organization since at least early 2020 (April 2020, October 2020, November 2021, March 2022), why is this only being announced now?
Edit with further questions:
Can you clarify the difference between NCRI the non-profit and NCRI, Inc.? What entities are these datasets being sold to?
Where are donations to the Pushshift Patreon going? Should users stop donating there and instead donate to the PayPal links you provided?
Comment deleted by user
ncri.io redirects to networkcontagion.us and has been used by SITM in the past.
account was added as a mod by u/Stuck_In_the_Matrix ~45 minutes ago.
That said the lack of communication continues, they popped in to confirm mod status but neither they nor jason has responded to any questions about the switch.
The account was added as a subreddit moderator by u/Stuck_In_the_Matrix
Taking this at face value till this is confirmed via any of the pre-existing channels.
Is SITM still handling maintenance? There are a large number of issues that need to be addressed.
I am but I've been away from comms this week and part of last week due to family issues. We're involving more people from NCRI to help with comms so that people aren't solely reliant on me at all times.
Thanks again for all your help s_i_m_s. We're working on a lot of improvements in our processes to avoid situations where people get frustrated from lack of comms or engagement. It isn't fair to the community even if I have valid personal reasons for not being able to respond immediately so this is a huge effort to improve on our comms and help fix issues reported by the community.