A Brazilian team used Discord’s API to scrape 10% of its open servers.

Researchers published a massive database of more than 2 billion Discord messages that they say they scraped using Discord’s public API. The data was pulled from 3,167 servers and covers posts made between 2015 and 2024, the entire time Discord has been active.

Though the researchers claim they’ve anonymized the data, it’s hard to imagine anyone is comfortable with almost a decade of their Discord messages sitting in a public JSON file online. Separately, a different programmer released a Discord tool called “Searchcord” based on a different data set that shows non-anonymized chat histories.

  • boatswain@infosec.pubEnglish
    131·
    1 day ago

    Seriously. It’s beyond painful when some open source project only uses Discord for communication. You have to hope that you post your question at a time when the right people are online, and that there’s not a more interesting conversation going on, otherwise it just gets lost. Index that whole dataset.

    • Ulrich@feddit.orgEnglish
      1·
      48 minutes ago

      Index that whole dataset

      I’ve seen a few projects doing just that with answeroverflow.com and they have come up in my web searches. Not really a solution but at least a stopgap.

      • The Quuuuuill@slrpnk.netEnglish
        49·
        1 day ago

        there’s a difference between using irc for livetime troubleshooting and not having a forum at all and directing everyone to your livechat discord. i’m sure some sicko out there has run an OSS project on only IRC, but their project likely got no traction because a history of problemsolving posts is important in open source. generally speaking, you need:

        • a wiki
        • a static indexable searchable forum
        • a live chat place for real time communication for novel problems

        too many projects these days only have that last one in the form of discord

      • AugustWest@lemm.eeEnglish
        9·
        20 hours ago

        For projects I am involved with all irc chats are archived and searchable. There is nothing private, no registration needed and searchable.

        Quite a bit different.

      • boatswain@infosec.pubEnglish
        11·
        1 day ago

        That would be equally annoying. Probably a better signal to noise ratio on IRC though; Discord descends into memes almost instantly.